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PREFACE 



During the 1974-75 school jear an ESZA Title Part C project m% 
conducted in North Carolina to Investigate the area of e\'aluation in 
local school districts and to develop, on the basis of this investigation, 
designs for evaluation systems for foLir of the State's local education 
agencies. During tlie y^ar^ siirveys of evaluation actwlties 1n school 
districts across the countrj were conducted. Visits were made to 13 of 
these districts to discuss in depth th€lr evaluation programs. Training 
sessions vvere held to acquaint the participants with current evaluation 
theor^^ and practice. 

All of the participants in the project were central office staff 
members of their respective districts. As the project progressed we all 
discovered that there was little In the way of literature or exaniple that 
v/ould assist administrators in fnaking decisions regarding the kinds of 
evaluation activities that could and sfiould be conducted in their districts. 
There appeared to be a need to pull together the information that was 
available and the inforination that v/e fiad gathered during the year for ' 
other administrators i^ho were making decisions concerning the type of 
evaluation activities that wuld be conducted in their districts. 

This document is an attempt to provide this kind of information to 

school district administrators. The document does not^ nor does it claim 

to present details or ''hov/-tos-' for specific evaluation situations. 

These details are more appropriate for evaluators and are addressed in 

much of the available evaluation literature. The contents instead j 

focuses on those factors about which administrators must make decisions 

In order to develop an effective evaluation program for their district. 

i 
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INTRODUCTION 

The time when administrators could choose whether or not thi^' would 
evaluate their school programs is probably behind us. Evaluation require- 
ments accompanying federal and now many state funds, state legislated or 
policy-based accountability programs and most modern approaches to school 
district adniinistration neccessitate the collection of evaluative infor- 
mation concerning many aspects of a district's programs and operation. 
Administrators are not, therefore, addressing the question "Will I 
evaluate?" but "What should I evaluate?, How do I establish a rnechanisni 
within the district to do evaluation?. What are my responsibilities for 
the evaluations conducted in the district?" 

It is the purpose of this document to provide information about 
evaluation which would assist administrators in responding to these 
types of questions for their district. It is addressed to adinini strators 
rather than evaluators for several reasons. First, as with other of the 
district's programs, administrators have the ultimate responsibn ity 
for the quality and appropriateness of the evaluations that are corducted 
by or for the district. Secondly, administrators, through their irdi- 
vidual personalities, leadership and responsibilities have a marked 
effect on many of the more technical aspects of evaluation. Stufflebeam 
(1974) and Sanders and Cuba (1973), for example, suggest that; (1) 
audience to whoni evaluation is to be addressed; (2) the sociopolitical 
envlronmenti (3) moral, contractual and legal requirements and consider- 
ations-, and (4) organizational and administrative structures, all elements 

I 
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of school district administration^ have a significant effect on such 
technical aspects of evaluation as: (1) the defmitian of evaluation 
employed;, (2) the tlrriing and means of initial ent^y into the evaluation 
process j (3) the arit&x^Ca that are brought to bear on the evaluation; 
(4) tN pm^iabUa that are selected for study; (5) the deaiffri that 
is develCDped; (6) the teahniqiwa that may be used; (7] the souram of 
infomation that may be tapped; and (8) the reports that are issued. 
Finally, most of the HterdLure concerning evaluation is addressed to 
the more technical concerns of evaluators rather than to thoseof admin- 
istratoirs. Little has been written that addresses more directly tha 
concenns of administrators who are assuming the responsibility of manag- 
ing and recti ng evaluation efforts in their school districts* 

The first five sections of the document deal with aspects of an 
evaluatloini program over which administrators have considerable^ if not 
a primary^ influence. These include:^ 

L CLIMATE: Developing a climate among staff and constituents 
that is supportive of evaluation 

IL PQCUS: Providing leadership in focusing the evaluation 

A, Assisting ir determining the meaning evaluation 
is to have in the district and the purpose{sj 
It 1s to serv^e 

B* Providing direction and Input into the design of 
evaluations to ensure that they meet the needs of 
the district 



Some of the responsibilities given in this list were taken from 
Thomas R. Owens, ''Suggested Tasks and Roles of Evaluation SpeciaHsts 
in Education," Educational Technology . XIII (1968), pp, 4-10, and Edward 
L. Morphet, Roe Johns and Theordore L, Reller, Educational Organization 
and Administratis (New Jersey: Prentice Hall, 196?), pp. 531^555. 
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III. ORGANIZATION: Providing an organization and support for 
evaluation within the district 

A, Establishing an organizational strLJcture for 
evaluation 

B. Establishing policies and procedures for the 
conduct of evaluation 

Providing the needed staffs materials, equipment, 
facilities and budget necessary to carry out 
planned evaluation activities 

IV, COMMUNICATION: Providing and maintaining open channels 
of communication among the administration^ school dis-- 
trict staffs -ocal board of education and public for the 
release of c ,aluat1on results and gathering of feedback 
concerning the results 

V. UTILIZATION: Providing the leadership and mechanisms 
for the utilization of evaluation results 

The sixth section provides a discussion of the role of the evaluator 
and the relationship of his or her role to that of , the administrator. 
The final section of the document discusses some of the Implications 
for administrators and school districts of evaluations and/or evaluation 
programs conducted 1n a district. 

It is hoped that after reading the document administrators will have 
a better understanding of the evaluation process as it applies to local 
school districts. It 1s also hoped that they will be more aware of the 
alternatives available to them and the implications of these alternatives 
for their particular school district. 
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L DEVELOPING AND MAINTArNING A CLIMATE 
SUPPORTIVE OF EVALUATION 



Administrators are perhaps in the single most Important position to 

influence the attitude and climate toward evaluation in their district. 

Dr. Jack Taylor, Superintendent of the Saginaw Public School views the 

Importance of this role In the follDWlng way: 

My relationship to the evaluation department is 
one of mutual dependence. I rely on them to pro- 
vide the kind of information necessary to make 
rational decisions and they rely on me, as super- 
intendent* to create a climate of acceptance for 
evalLiation procedures among the professional staff 
and especially among the administrative decision- 
makers* (Taylor s 1975s p. H) 

An administrator Influences the staff's attitudes (1) by his or 
her action or Inaction in exerting leadership in evaluation activities; 
(2) by communicating the degree of importance which he or she attaches 
to evaluation in the district; and (3) by the manneri i,e, punative or 
constructive. In which results are used, To establish and maintain a 
climate supportive of evaluation In the district the administrator needs 
to communicate to the staff the Importance he or she places on evaluation 
and to clarify the manner in which results will be used. 

While each adirilnistrator knows the best way to obtain the cooperation 

and support of his or her staffs it may be of some benefit to identify 

2 

some activities which may lead to a climate supportive of evaluation. 

D§tBwiining the pot&ntial reoipientB^ arydi/ov targ&t 
gpaupB^ existing atti^de tojDo^d evaluation ar^ aon^ 
amtrating effortB to imppove the attitudea of thoss 
whc ore mo8t negative* 



Again s some of these activities were adapted from the Owen's article. 
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strategies as formal as the administration of instruments designed 
to measure attitudes toward avaluation or as informal as^ casual conversa- 
tions betv/een administrators and their staff can be utilized to determine 
attitudes toward evaluatlori. Based on the results of these surveys 
administrators can plan more specific activities such as those that are 
described In the refnalnder of this discussion to gain support for evalua- 
tion in the district. 

3i SmttCng an emample of suppoi't by aoting positiv&ly 

upon tJim reaeipt of hoth positive and negative i^bbuHs, 

The example that administrators set when they receive evaluation 
results has a great Influence on the way In which the staff receives and 
uses results. Administrators who view favorable results as a reflection 
of the quality of the staff and the district's programs and view unfavor- 
able results as valuable irforfnation upon which to base Improvements can 
favorably influence the staff's receipt and use of both favorable and 
unfavorable results. 

5* B&tabliBhing evaliiation as an integral paTt of sahool 
dietiHat ap&Tation far acbnini8tr^ativ& and tnstvuQt%onal 
drnGimion-makirLg arid TspoY^ting sahaol achi&uement to the 

When evaluation is utilized regularly, the fear of evaluation 
produced by unfamili ari ty and uncertainty can be lessened 
considerably » Mminlstrators can best achieve this acceptance by estab- 
lishing a continuous program of evaluation that pervades all levels of 

school district operation. 

4. Staiinrj Qtmarlii thm purposQs for* matuation and the 
rol& of eaah &baff member in oonduating and using the 

In addition to creating farniliarity through experienca, an administrato 
can decrease the uncertainty associated with evaluation by creating 

a conrimon understanding among his or her staff of the meaning and purpose 



of evaluation and the role eacfi staff member is to perfafm. One way of 
achieving this goal which is highly recommended is the establi shrnent of 
district wide policies and procedures for evaluation which cam then be 
conmuni cated to each staff member, 

5. Securing the paT-biaipation of staff in ptannCng^ 
■imptmmntation and use of Bvaluat'ion, 

Greater support for evaluati on among the staff can be obtained if 

they are not solely the victims of evaluation efforts but are also actively 

InvoWed in decisions regarding the evaluailon to be conducted. Staff 

participation can be helpful to tdnil nistrators and evalijators alike In 

deteriDining evaluation needs, designing evaluations to meet these speciffc 

needs and providing suggestions and assistance 1n the use of results. 

6. Nainta-Cning open ahannets of oommn'Laati.on heWsen 
the adminiBtrabianj evaZuatorSf staff ard publio. 

The fulfniment of this rnajor responsibni ty can do much to ease the 
minds of those persons dlrectli' affected by evaluatloti. As we as a nation 
have so recently d1 scovered, siippression of information and Isolation of 
leaders In regard to any given prograni can lead to paranoia^ fear» dis- 
trust and a variety of other urdeslrable situations and environments. 
Admiriistrators who establish an open comnnunl cation policy regarding evaloa- 
tion and take steps to accurately portray the results that are obtained 
can do much to reduce the incorrect rumors and suspicions that so often 
accompany' the innocent or overt suppression of Infovmatlon from constituents. 

7. Piravid'ing administrat'tve support and aeeistianae fro 
staff whose ppajeata or per*fonmnae ave bsi-ng evaluated. 

In order to develop support for evaluation, staff members need to 

know that they do not become "untouchable" or outcasts when their projects 

and/or perforniances are being evaluated. If the purpose of the evaluation 

is improvementj this fact should be clearly communicated to the staff and 
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the necessary support services to enhance improvemeiit should be made 

irnmedlately and continuously available. In cases where evaluatton 1s to 

be utilized to fnake decisions regarding continuation or termination of 

projects or ernployment^ this situation should be most clearly coimnLjnlcated 

and thoroughly explained to the staff. 

8^ Foausing mvaZuations as the infomattan n&eds of ths 
dis-tznat staff and/c^ aonst%tu&ntQ* 

Perhaps the greatest support for evaluation can be obtained if staff 
fnembers and constituents are provided Information whi ch they perceive to 
be of inmedl ate values appl icabillty, and interest* This requires that 
administrators ba cognisant and sensitive to the needs of their staff and 
public and provide evaluation services that v/ill meet these needs. 
9. Providing staff insevviam m mvaZucLtion. ' 

Tha capability and willingness of staff members to participate in 
planning and utilizing results of evaluations should be increased as they 
learn more about the process of evaluation. In-service training In the 
evaluati on process 5 e.g. Identifying evaluation needSi planning evalijatlons , 
selecting^ developing, and administering evaluation instruments^ analyzing 
results and developing reports describing evaluatloris ^ should give the 
staff sufficient skills to actively participate in planning and conducting 
evaluation activities. In-service training sessions accompanying the 
release of evaluation results should not only describe the results but 
also contain suggestions that staff fnembers could follow to use the evalua- 
tion results in their work. 

In sunmirys the climate or attitude toward evaluation has a great 
deal of Influence on the way and extent in which evaluation results are 
used 1n a school district. Administrators wishing to inaximize the use of 
evaluation results need to be aware of staff and constituent attitudes 

a 
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toward evaluation and take actions to improve or maintain a supportive 
climate in which to conduct evaluation activities. 
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n. FOCUSING THE EVALUATION TO MEET SPECIFIED fiEEDS 

The potertial usefulness of avaluation 1s deterfnined^ to a lai-ge 
degree, by the adequacy with which it addresses the needs of ir^dlvlduals 
or Institutions who v/m receive the results. One of the first evaluation 
tasks is^ thereforep one of determining what the needs for evaluation are 
and then selecting the kinds of evaluatiye Information that win meet 
those needs. Administrators of school districts are usuallj^ 1r the posi- 
tion of knowing the mst about the district* s overall needs. It Is, 
therefore, important that they participate in the decisions that lead to 
a narrowing or focusing of evaluation efforts* 

From an administrative viewpoint^ some of the more Important categories 
of alternatives which provide a focus to evaluation are: (1) the meaning 
and purpose given to evaluation; (2) the specific audiences that the 
evaluation will serve or address; and (3) the information that the evalua- 
tion will provide. It is important to reiterate that these are categories 
of alt&x'miivs8> There are, in other words, . ..no itmiiedlately correct sets 
of variables or, designs for evaluation* Each alternative gives evaluation 
a rather specific focus and should be weighed in light of the particular 
needs of the district. 

Determining the Meani ng and Purpose of Evaluation 

One of the first steps In focusing an evaluation is to establish a 
clear position regarding the meaning and purpose evaluation is to have. 
Because educational evaluation evolved from a variety of conceptiial 
bases and has been utilized for many purposes^ 1t is quite common to find 
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mr\^ definitions and uses for evaluation in the literature and In practice. 
Etch of these approaches has some definite and often different implications 
for the manner in whicli e'valuations v^ill be designed, conductid and used. 
It, thereforsi becociies important that administrators seeking to meet 
certain needs know v/hat th&y can expect to obtain from the approach they 
select. 

What is EvaLlu at1jn?_ 

/appraisals are inavi table, Citizers, parents, students^ 
teachers* administrators* board of education members^ and 
representatives of the state department of education have 
views ( JudgfTients) regarding the strengths and limitations 
of gi\^en schools or school systems,, ,The question that con- 
fronts the educational administrators therefore^ is not 
whether or not there wiTl be appraisals, It is, ratiierg 
whether or not the appraisals will be reasonably valid or 
only judgments made on the basis of inadeqiiate data, or 
even v^ith merely rumor as the foundation J' 

(Morphet, Johns and RelTer, 1957^ p. 533 

Evaluation* as perceiyed for the purposes of this paper, is not a 

detemii nation of worth which is based solely on perscnal perceptions and 

opinions s but a disciplined process of inquiry deslgred to obtain accurate 

Information about its objects. Evaluation of a school district's r€ading 

program, for exaniplei would not be considered adequate or complete if 

taken from stateinents in a newspaper editorial i remarks of a vocal citizens' 

group or even opinions of the district's administration. While each of 

these "evaluations" may in reality be correct, the type of evaluation which 

is to be discussed here 1s a more systeinatic and scientific process which 

should lead groups having divergent opinions to similar conclusions if 

the process were repeated^ 

Steele (1973)^ for example has categorized over 160 approaches to 
evaluation into six inajor and different conceptual areas. 

12 
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EXAMPL E I 



In a recent telephone survey of State Education Agencies 
the question was asked, "Would you identify sofne school 
districts 1n your state which are conducting exemplary 
evaluation programs?" Examples of the responses given, are 
1 isted below: 

1. *'A11 the districts in our state conduct evaluation 
programs. Each has an extensive program of group 
and diagnostic testing*" 

{Eoatuation means te&t^ng) 

2. "There are no districts 1n our state that have an 
evaluation prografn." 

{Mmv& it is diffiault to determine what evaluation meant 
to ike T&Bpond&nt beaw&e thm our^Tent stats education 
di'^ectoTy tiBtmd fovz^ sa}^Ql diatTiatB tiiat had dir&ators 
and/or d&partmentB of smtuation* } 

3. "We are just beginning to v/rite objectives in many 
of our school districts. We hope to begin Imple- 
menting evaluations of these objectives next year J' 

{Valuation meana d^t&mining the aahi&vem&nt af cbj&a^ 
And the most comnion response: 

4* "What do you mean by 'exemplary evaluation programs*?'* 
{M^atuation m&ansff'J} 

In spite of this apparent confusiDn, there are three definitions of 
evaluation that currently seem to be receiving the most widespread recog- 
nltiofi and use. They are: (1) evaluation 1s the process of determining 
the actiievement of objectives; (2) evaluation is the process of providing 
inforniation for dec1sion--niaking ; and (3) evaluation is the process of 
determining the worth or merit of a program, activity ^ etc. 

The definition which has probably received the greatest acceptance 

and use among educators in local school districts is one in which 

evaluation Is defined as the process of aornparing pBTfomanae with 

beJmvioTalty atat&d objmtivm. This approach to evaluation was 
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developed in the 1930 's by Ralph W.. T^'ler (1942) and his associates 
as they conducted the well -known Eight-Year Studi^. While variations 
on this definition have been developed over thei/ears^ the basic 
procedure for conducting evaluation when this definition is used 
is to: (1) develop a set of behaylorallj stated objectives; (2) select 
or develop measures to deterinine the achievement of the Qbjectlvesi 
(3) gather data on the specified behaviors; and <4) compare the results 
to the pre-spec1fied criteria set in the objective i A prograiri is then 
Judged on the basis of the achievement of its abjectlves* For many 
evaluations of this typep the objectives reflect desired student outcome 
behaviors which are viewed by fnany as the ultifnate criteria for program- 
matic success, * ^ 

There are several advantages to this approach to evaluation. Firsts 
basing evaluations upon pre-specif ied objectives provides an Imniediate 
focus and thus simplifies the process of evaluatiori design. Some objec- 
tives are even written so that complete specification of the evaluation 
to be done is included in the objective itself. Secondly, because the 
criteria for success is stated in the objectives it is easy to determine 
if the objective has been achieved. It is simply a matter of comparing 
the evaluation results to the stated criteria. Finally ^ objective based 
evaluations focus on outcomeSp usually related to stiidents, which are very 
Important concerns for school districts and their corstltuents. 

There are, howeverg some limitations to the use of this definition at a 

school district level that should be noted. As mentioned previously, most 

objectives are written in terms of student outcofne behav^iors. Evaluations 

based on these outcomes provide some very important irformatlon related to 

student status and progress. They do not, however, provide information 

on the aspects of the program such as teaching methodi content, cost^ suppor 
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services, etc. whlcii an admlm'strator must manage in order to correct 
deficiencies ir student outcome behaylors or other rilatid student variables 

Such as sodo-econoiiii c statiis, home environment, etc. which tn&y have consider- 
able effect on student outcome .behaviors. These evaluations, 1n other 
words, point to tha illriess but do not suggest causes or cures. 

A second problem is that the development of objectives often breaks 
the prograin into pa.rts , e.g. achleyement and attitude, which are then 
evaluated separately- This fractional 1zati' on makes it difficult to view 
the program as a viliole or to determine the relationship between the behaviors 
ineasure. 

Finally, the basis for conducting this type of evaluation, is the 
existence and qualtti' of a set- of objectives. Programs for which no 
objectives are developed or are readlli^ discernible are difficult, if not 
impossible to evalua-te using the steps listed earlier. In addition, 
because the objectfvfis themselves are seldom evaluated, the quality of 
the evaluation must most often rely on the quality of the objectives that 
have been developed. Progranis vjith v^eak or inappropriate ohjectives will 
most often produce weaLk or irapproprlate evaluations. 

A second popular definition of evaluation is providing infomation 
for dma^sion'maJdng. Administrators adhering to this type of definition 
would be especially sensitive to the decisions that would need to be made 
about a program and see to it that Information related to the decision 
Is mde available through the evaluation, This definition has been par- 
ticularly beneficial in focusing the use of evaluation to the educational 
and administrative questions that are daily faced in schools. It also 
has the advantagB of expanding the kinds of evaluative information that 



can be provid^jd to includi information about aspects of a prograni such aS/ 
needs* goals and objectives^ procedures and activities as wall as outcomes. 
Finally, this type of evaluation has the adyantage of providing Information 
at any time in a program's 'operation that a decision is needed rather 
than only at the endrof a program when outcomes are determined. 

The major limitation o.f this type of evaluation Is the difficulty of 
Identifying and specifying educational decisions and decision-makers. 
Without this Information it becomes difficult to design appropriate evalua- 
tions to meet decision making needs. A s€cord limitation occurs when 
attempting to distinguish between the kinds of Information that are 
needed for decision making. The meaning i/^hlch e/aluatlon is to have 
becomes somewhat confused when information wlilcti is not clearly evaluatwe 
is provided for decision making purposes, Fimally, providing information 
for many of the educational decisions that need to be made can be an 
extremely complex and costly process^ decreaslrig its capability of being 
implemented in many school districts. Because of these and other diffi- 
culties associated with the use of this definition^ some evaluatorss e,g. 
Scriven (1967) and Stuff! ebeam (1975), are perceiving It as a purpose or 
use for evaluation rather than a statenient o-f nieanlng. 

A third definition which 1s gaining iricreasi ngly wider acceptance 1s 
the one provided us by the dictionary. lr\ this definition^ evaluation is 
the process of detemining the worth or mmi of a thing, Clearlyj this 
is the simplest of the definitions given tiere and the least specific In 
ternis of providing Indicators of how the evaluation is to be done. It does^ 
Nowever, have the widest acceptance and understanding by educators and the 
lay public alike and appears to make up in meaning for what it lacks in 
specificity. The use of this definition requires that information be 
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gathered, analyzed, and interpreted so that a determination of the merit 
of the prDgrani or any of Its aspects being studied can be made. Because 
it is not specifically tied to student outcomes or Instructional decis'ionSs 
all areas of the school program can be considered when an avaluation design 
is being developed. The selection of an evaluation design or techniques 
when this definition is accepted^ becomes more closely aligned to the pur- 
pose for v/hich the evaluation 1s being conducted and the specific questions 
that it is to address. 

The major limitation of this definition for administrators is its 
lack of specificity for designing and conducting evaluations. Persons 
using this definition for evaluation must use their knowledge of the many 
techniques of disciplined inquiry in order to develop an evaluation 
design that ^111 provide information that can be used to determine the 
worth or merit of the object of the evaluation. A second limitation is 
the difficuUyof determining the criteria or values that will be used to 
determine wrth or merit* 

Unfortunately I none of these definitions has become universally 
accepted and all are being heavily utilized. It 1s , therefore, important 
that administrators become acquainted with existing conceptualizations so 
that they can (1) overtly select a definition which best fits the evalua- 
tion needs of their district; (2) best use the results of mandated evalua- 
^ tions whicJi may be based on a differing conceptualization than the one 
used in the district; and (3) provide a common basis for comuni cation 
concerning evaluation among their staff and constituents. 

What Is The Purpose Of Evaluation? 

While there are many purposes for which evaluations can be conducted 
in a school district^ it appears that they all cluster into two main 
categories: (1) providing information which is relevant and useful to the 
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decisions that are made In the district; and (2) providing information 

on school district accomplishments for distribution to various funding 

and/ovr governing agencies and publics. 

The major characteristic of evaluations that are conducted to provide 

infarTOtion for decision-making is their responsiveness to evaluation 

needs wMdn the district, are designed primarily to provide 

infomtion to use in the administration of schools and the Improvement 

of aducational programs* They can be designedi for eKample^ to: 

1. determine the effectiveness of instructional programs; 
stafft raethods, etc; 

identify operational needs i 

3. provide a basis for the allocation of funds to programs 
and/or operational areas; 

4. Identify strengths and weaknesses in a given program area; 

5. determine the worth of alternative approaches; and 

6. monitor ongoing programs, 

When the evaluations are completed they are fed back to the staff 
personnel in the district who are responsible for the area being evalua- 
ted. Evaluation, for this purposei is most often conducted and used 
Internially by the district. 

Evaluations which are Intended to provide infonnation about school 
accc^mplishnients to an external audience are characterized by a more out- 
cow^-Oriented approach and, in many cases, adherence to the design and 
reporting regulations of the agency to whom the report is sent. These 
evalueitions are used primarny as a school public relations and partici- 
pation tool between the district and its comnunity and as an accountability 
mechar^lsm for the constituents and agencies which have granted funds or 
servlcss to the district. They can be used^ for example, to; 

18 



1, provide Information on school achievements and/or needs 
to local decisfon-making bodiesj 

2, provide information on school achievements and/or needs 
to the pub! 1c J 

3, meet state and federallj? supported program requirements; 

4, comply with legislated accountability requirements; 

5, comply with state accreditation requirements i and 

6, comply with regional accreditation requirements. 

Evaluations conducted for this purpose have a more external use with 
the results most often being transmitted to groups or agencies outside of 
the Immadiate school district staff. 



EXAMPLE B 

In the Dallas* Texas school district, specific 
provisions through board-approved policy have 
been developed to ensure the appropriate use of 
evaluation for both decision-making and accounta- 
bility, ProaesB evatuaHonB^ as they are called ^ 
are conducted by the district *^s evaluators to 
determine specific strengths and weaknesses in 
program operation. These results are^ by policy, 
relayed only to the program staff members who 
have the responsibility for implementing and 
directing the program. Based on the results^ the 
program staff can a Iter or modify aspects of the 
program that are not effective and capitalize 
on Its strengths, ppoduat mvaluations are con- 
ducted to determine the overall accwipllshments 
of the program and are, by policys reported to 
the administration and local board of education. 
In this way evaluations are tailored to the needs 
and responsi bill ties of both the program staff 
and district-wide dectslon-makers. CWebster, 1975) 

When the Information needs of external agencies and the public over- 
lap with t^ose of the school district staff, evaluations can be designed 
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to serve both the purposes of decision-making and reporting to external 
audiences. When the information needs differ, however^ choices need to 
be made concerning the purpose of the evaluation so that appropriate 
questions can be studied and reporting mechanisms developed, A quick 
check of the purpose for which an evaluation is to be conducted can be 
made by comparing the district's purposes and the characteristics of its 
use to the chart in Figure I. 

Figure 1. A Comparison of the Two Major Purposes 
for Evaluation in Local School Districts 



Purpose 


Provide information for 
administrative and instruct 
tional decision-making In 
the district 


Provide information on 
accomplishments to fund in 
governing agencies and 
the public 


Goal 


Improved instruction 
More efficient district 
operation 


Improved school /comniunlty 
relations 

School district accounta- 
bility to funding/ 
governing agencies and 
public 


Focus of 
the Evalua- 
tion 


Aspects of school district 
operation which need to be 
improved or strengthened 


Total school district and 
or individual program 
accomplishments 


Primary 
Audience 
for the 
results 


School district staff 


External funding/govern In 

agencies 
Public 


Major cri- 
teria for 
judging the 
evaluations 


Relevance to school district's 
needs 

Timely provision of information 
Responsiveness to district 
evaluation needs 


Objectivity and 
Credibnity of results 
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Identifying The Audiences For Evaluation Information 



Closely related to determining the purpose for evaluation is the 
identification of the audiences who will receive^and use the resulting 
information. This is because one of the key characteristics of effective 
evaluation is its utility or capability to be used. In order to be of 
use^ evaluations must be focused to a large extent on the needs of their 
audiences* 

The extent to which the results of an evaluation are perceived to 
be useful to any group is largely determined by the degree to which the 
results address areas of Interest and usefulness to them, Kor an instruc- 
tional program area, for example^ teachers may wish to know which materials 
and approaches are most effective for students of various abilities. 
Principals may wish to know which organizational patterns, i.e. team teach- 
ing, open space, traditional, etc.* are most effective. Supervisors may 
wish to identify areas of needed teacher In-service* The administrative 
staff may wish to know the overall accomplishments in the area and be 
provided information that would assist them in allocating personnel, 
materials, facilities and monies to the program. The board of education 
and public may wish to know what students are achieving in the selected 
program area. 

It is for this reason that administrators and evaluators need to 
identify the major audiences that each evaluation will address. Provi- 
sions can then be made to ensure that the design of the evaluation 
Includes questions addressed to the types of information that its audiences 
win use. 
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EmrpLE 3 



Some of the audiences to which a school district might 
direct Its evaluation results include: 

DIRECT PARTICIPANTS 

Local Board of Education 
Superintendent 
Administrative Staff 
Instructional Supervisors 
Principals 

Department or Grade Level Chairpersons 
Teachers 

Instructional Support Staff 
Noninstructional Support Staff 
Students 



INDIRECT PARTICIPANTS--CLOSE ASSOCIATION 
Parents 

Ancillary Booster Groups 
Local Education Organizations 

Teacher Organization 

Phi Delta Kappa 

Special Interest Parent Groups 
Education Committees of Larger Organizations 

Service Clubs (Lions^ Women^s Club^ etc.) 

League of Women Voters 
State Education Agency 
Federal Agencies 
Foundations 



INDIRECT PAR TICIPAN TS--DIST ANT ASSOCIATION 

Pol Itlcal Leaders 

Business and Industry Leaders 

Community Agencies and Organizations 

Professional People 

"Blue Collar" Working Class 

Disadvantaged People 

Non-Parent Adults 

Older Citizens 

Nearby Colleges and Universities 
Educational Researchers 
Media 

{From Kean (1975) ^ Woods (undated) , 
and Bettinghaus and Miller (1973)} 



IdentiFying the audiences to whom evaluation results should be 
addressed determines who will be the target of the results but does not 
specify what information they need. To determine the information that 
should be transmitted to identified audiences requires an awareness of 
their interests in educational informations their knowledge of potential 
areas to be Included in the communication and their attitude toward the 
schools and the particular program being evaluated. 

Administrators know as a result of their experience that the various 
groups associated with their schools have differing interests in the school 
program. Some groups are most interested in the instructional programj 
some the achievements of pupils, others the costs for education^ etc* If 
the major interests of targeted audiences can be identified^ evaluations 
can be focused more appropriately to their particular interests and needs. 

EXAMPLE 4 

McClosky (19675 PP^ 244-245) pfor example^ provides a chart 
of typical publics ^ size and comnunication function and 
major interests 1n schooling which may be a helpful starting 
point for district personnel who are trying to identify the 
interests of targeted audiences^ Information such as the 
following is provided* 

Public 

Parents 

Size and Communication Functions 

Largei exceptionally interested grpup^-includes some social 
leaders^-constantly communicates widely through person-to- 
person and person-to-group channels. 

Major Interests in Schooling 

Children's educational needs and accomplishments^ school's 
contributions to ch11dren*s well-beingp happiness, and 
social status^-personal satisfactions derived from children's 
academic achievements and participation in school activities. 
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As administrators identify the audience for evaluation they will, 
undoubtedly^ on many occasions find that there are several audiences which 
would benefit from the results. To the extent that it is possible .evalua- 
tions should be designed to meet these different needs. When the extent 
of the evaluation effort becomes prohibitive or the needs of the various 
audiences conflict* administrators should then place these audiences In 
some priority order so that the needs of the most important or crucial 
audiences can be met. 

What Information Should The Evaluation Provide? 

The areas of an educational system that can be evaluated are limited 
only by the techniques and skills which we currently possess to evaluate 
them and the Imagination of those responsible for selecting areas of the 
school program to be evaluated. All aspects of school district operation 
in effect are^ therefores available for initial consideration as an 
object of evaluation. The difficult tasks are those of: (1) selecting 
those areas of the school district program where the need for evaluative 
information is the greatest; and (2) determining the types of information 
that are needed about the areas that have been selected. 

Selecting the Areas of the School Program to be Evaluated 

Before selecting areas of the program to evaluate , administrators 
should, from some framework, consider all of the areas of the district 
that might benefit from evaluative information. There are many ways to 
view the facets of a school district that can be evaluated. One helpful 
delineation is provided in the partial list of appropriate evaluation 
objects given by Stuff! ebeam (1975 5 p* 8); 
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Programs - ongoingi goal directed activities, 

involving especiany curricular offerings 

Projects - time-bounded, goal-directed activities 
such as federally funded experiments 

Personnel - teachers, administrators, coaches * 
counselors, custodians, etc* 

Students - persons enrolled 1n instructional 
offerings 

Facilities - buildings, equipment 

Materials - books, filmsi tapes ^ etc. 

Ideas - theories, planSj and designs 

Institutions - colleges, school systems, community 
service agencies, foundations , etc. 

Evaluations - evalution studies themselves should 
be evaluated 



Using this breakdown it is possible to envision the evaluation of: 
(1) regular, on-going programs In a school district such as the reading, 
physical education and cultural arts programs; (2) special local and 
externany funded programs such as experimental approaches to classroom 
organization e.g, open classrooms, team-teaching. Individualization of 
instruction, and ESEA Title I and Title III programs; (3) teachers, 
paraprofessionals, administrators, supportive staff; (4) student progress 
in cognitive j affective and psychomotor areas ; (5) the adequacy of buildings 
and equipment for programs which have specific facility needs^ e,g. 
vocational education, environmental education, science, physical education, 
etc; and so on. 

Another useful way to categorize the kinds of evaluation that might 
be conducted is according to the major areas that comprise school district 
operation, e*g* administration, curriculum and instruction, personnel^ 
finance and business management, and physical facilities. 



EXAMPLE S 



The Philadelphia public school system describes its 
evaluation focus around some of the major areas of 
school district responsibility, 

1 , Administrative and Survey Research Services: 
provides a pupil data systenis socioeconomic 
and demographic analyses about the school and 
community populations, followup studies of 
graduates and dropouts , and maintains a manage- 
ment information center for administration. 

Z. Instructional Research an d Development Services : 
assists in the development of proposals and tests 
and maintains a data management unit of instruc- 
tional information- 
s' Testing Services : has the responsibility for the 
development. Implementation, and administration 
of all phases of the district's group testing 
program, provides staff in service related to 
testing. 

4. Priority Operations Evalu ation Services: provides 
research assistance in the priority areas of the 
district superintendents and evaluates programs 

in the five priority areas of the district. 

5. Feder al Evaluation Resour ce Services: conducts 
evaluations of public and non-public school 
federally funded categorical projects, 

(Kean, 1975) 

A third way in which to view the kinds of programs that can be 
evaluated is according to the types of evaluations that are conducted, 
e.g. surveys, program evaluation, testing or assessments research, etc. 

EXAMPLE 6 

SaginaWj Michigan public schools categorizes its 
evaluation according to the types of services that are 
performed. Some examples of the areas included 1n 
their program are: 

I . Program Evaluation 

1. Evaluation of ESEA Title I proqrams 

3, Evaluation of the adult basic education program 

4, Evaluation of the migrant education program 

5, Evaluation of the b1-lingual education program 
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II . Assessment Services 

8. Provide coordination and supervision of the 
school district's assessment program 

10, Develop and maintain the department's 
Instrument Bank 

III* Research Services 

11, Conduct a dropout study in each se^condary 
school 

12, Prepare a written report of the student 
attitude survey conducted 1n grades 8 and 11 

15. Develop a mechanism for disseminating current 
research findings to the professional staff 

17. Conduct a context evaluation (needs assessment) 
at the district and building level 

(Taylor, 1975) 

Utilizing these or other frameworks for reviewing the district's 
programs, administrators should quickly find that there are many more 
areas that could be evaluated than time, staff and other resources allow. 
The selection of areas which will ultimately be evaluated 1s most often 
then a matter of narrowing the choice of alternatives by determining the 
Itnportance to the district of the resulting evaluation information. 

There are a number of ways In which the importance of evaluation of 
given areas of the school program can be identified. Whether planning for 
the development of a comprehensive evaluation system in a district or for 
a single evaluation, a good place to begin ls with the information which 
is required by 1 egi si ative, funding and policy mandates and, therefore, 
must be collected. There is no decision to be made here, but the identi- 
fication of this information provides a list which can be consulted when 
other Information needs are identified. Reference to documentation of 
required evaluation information helps planners prevent the duplication of 
data gathering activities and helps them judge the amount of further 
evaluation activity they can afford to conduct. 
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It is a little more time consuming to select the remaining areas of 
the school program that need to be evaluated. Several sources can provide 
inputs into this decision. 

One source for determining the area to be evaluated is the opinions 
of the staff and constituency of the school district. Surveys asking 
persons to list their preferences for evaluation or prioritize a list of 
evaluation needs can be administered to the total school district staff, 
special subgroups of the staff , local board of education members 5 repre-- 
sentatives of key coniTiunity groups^ the total community, parents and/or 
students. Results of such a survey can provide a list of evaluation 
needs 1n order 0^ their preference* 

Another source for identifying areas that need to be evaluated are 
those areas of the school program which can be identified as being par-- 
ticularly weak or strong. Particularly weak programs need to be evaluated 
so that problems associated with the program can be Identified and correc- 
tive plans developed. Particularly strong programs need to be evaluated 
so that the reasons for success can be identified and, 1f possible, utilized 
in other programs. Programs that need Intensive evaluation can be Identi- 
fied by reviewing test scores and other evaluation data^ consulting with 
teachers and supervisors or by obtaining suggestions of the administrative 
staff. 

A third source of information that can be used in identifying evalua- 
tion needs is the new and/or Innovative programs designed to meet a special 
need for which there is little or no evaluative data. Because they are 
new to the system s evaluations can be designed to determine if the programs 
were correctly and fully implemented and if they are achieving the desired 

results, g5 
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Another source to consider when attempting to determine what will be 
evalgated is thn specific interests of the local board of education members, 
parents 5 comnunity groups , etc. Responsiveness to these needs can con- 
tribute to positive school "Community relations and to support for the 
evaluation program in the community. 

After the most important areas for evaluation have been Identified, 
it may still be possible to have too many areas of concern to feasibly 
evaluate* In these Instances administrators may need to seek ways to 
prioritize these areas or set up a multi-year program to evaluate each 
of the selected programs* 

WXMfPLE 7 

The White Plains, New York school district determines 
what it will evaluate on a yearly basis. In the early 
spring requests for program/project evaluation are 
solicited from groups such as the building principals^ 
the PTA council * advisory coninlttees ^ teachers, students, 
parents and the administration. An Evaluation Priority 
Committee composed of the assistant superintendent for 
instruction, members of a district coordination team 
for curriculum and^ instruction, the superintendent's 
administrative assistant, the research/evaluation coor- 
dinator, the reading coordinator and the assistant director 
for pupil personnel services reviews the list and the 
district's required evaluations (federal programs^ 
Board of Education goals, etc J and develops a proposed 
list of evaluation priorities. In June, this list is 
presented to the Administrative/Supervisory staff of 
the district for discussion. On the basis of this input 
and the resources available in the district for evaluation, 
the Evaluation Priority Cofmittee determines what will be 
evaluated during the upcoming school year* 

(ReconTnended Process for Identifying 
EvaTuation Priori t.l|5_j 19^ 



Determining the Type of Evaluation Information that Is Needed 

Once areas to be evaluated have been selected, the specific types of 
information that need to be gathered should be determined. When, for 
example, It has been determined that the area of reading should be evaluated 
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administrators and evaluators need to decide if the evaluation design 
should contain provisions for gathering data related to student achievement, 
student ability, student attitudes, student interests, teacher preparation, 
adequacy of materials, adequacy of Implementation of particular programs, 
etc. 

At this point it might be contended that administrators who tread here 
would be assuming a role that rightfully belongs to the evaluator. In 
some cases, such as an external evaluation of a school district, this may 
be true. In other cases, however, when the district itself is seeking the 
evaluation, administrative input, or at least review of the types of 
Information to be gathered sit is important to ensure that the evaluations 
meet the d1str1ct*s need. 

There are several sources which may be considered to determine the 

types of evaluation Information that might be collected: 

K the types of information Identified in objectives 
related to the program^ 

2. the particular purpose for which the evaluation is 
being conducted; 

3. the audiences who can be expected to be interested 
and/or use the results; 

4* the Qpinions of experts and/or research in the 
area being evaluated; 

5, the types of information specified In an adopted 
model for evaluation; and 

6. the experience of others who have conducted similar 
evaluations* 

Objectives for the program being evaluated, whether specifically 
developed by the district or drawn from curriculum guides, textbooks ^ 
policy manuals, etc* provide some information on the types of behaviors, 
e.g. achievement, attitude^ etc. and activities that can be expected In 
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a given program. These are important first sources for persons designing 
evaluations to consult. 

If the purpose for the evaluation is decision-making, there are 
questions such as the following that need to be answered. What kinds of 
decisions need to be made? Who will make the decisions? What types of 
information do they need? If the purpose, on the other hand, is to report 
the results to external audiences, what types of information do the 
recipients of the results need? 

If audiences have been previously identified, it may be helpful to 
refer their particular interests and information needs as a source of 
information for determining the types of data to collect. 

Experts in the area are always a good source of Information to 
consult when trying to determine the types of Information that are needed 
to evaluate a program. Subject area specialists are often found on the 
school district staff and can be consulted to determine the types of 
information needed. Other experts can be hired or consulted through their 
writings. The library is always a good place to begin a search for the 
types of information needed because It contains the opinions of experts, 
reviews of the literature on the topic and research studies which have 
been conducted to determine the important factors related to the topic 
being evaluated. 

Some administrators may find it beneficial to adopt a specific model 
for evaluation which Identifies some of the types of information that 
should be gathered. If, for example, an administrator were to choose to 
utilize the CIPP (Stufflebeam, et.al., 1971) model for evaluation, he or 
she would seek information related to the goals of the program, the parti- 
cular design of the program, its implementation, and the results achieved. 
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If a model such as the one presented by Hamniond (no date) were chosen ^ 
types of information gathered would be related to behavior (cognitive, 
affective, psychomotor) ^ instruction (organization, contents method, 
facilities, cost), and institution (student, teacher, administrator, 
specialist, family, comnunity). There are, of course, other models for 
evaluation that could be chosen, each having some implications for the 
type of information that should be collected. 

A final source for determining the type of evaluation to be gathered 
is the experience of others who have conducted similar evaluations. Per- 
haps the largest source of information related to the experience of others 
is the ERIC collection which can be found in most major libraries and 
contains many evaluation reports conducted in school districts. A com- 
puter search of this collection Is rather inexpensive and can provide 
abstracts of the evaluation reports concerning a given topic that are in 
the collection. Another source is either direct contact or the reports 
of personnel in other school districts who have conducted similar evalua- 
tions , 

EXAmLE 8 

A study of evaluation designs in Portland, Oregon 
public Schools revealed that the best designs asked 
for the following types of information. 

1* Are the goals of the project being evaluated 

consistent with the program goals of the district 
and/or area in the subject area concerned (e*g, 
reading instruction!? 

2. What are the relevant characteristics of the 
student population receiving instruction via 
the method or using the product in question? 

3. To what extent is the program successful In 
achieving its goals? 

4. Is the program economical to operate when com- 
parisons are made with other programs having similar 
goals? 
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5. What special resources, personnel and facilities 
are requirid to successfully operate the program? 

6. How does the efficacy of the project vary 1n 
relation to the characteristics of the schools in 
which It Is operating? 

7. How does the efficacy of the program vary in relation 
to the varying characteristics of the student popu- 
lation (e.g. sex, age, aptitude, ethnicity, socio- 
economic factors, etc,)? 

8. How well is the project functioning in relation to 
the operational plan eftablished for It? 

(Hansen, 1975) 

While it is probable that a great deal of assistance can be provided 
by an evaluator in determining the types of Information that need tu be 
collected, it is important that an administrator be cognizant of the types 
of Information that he or she needs to have to obtain a useful evaluation 
of the selected program. 

In summary, there are many factors to consider which affect the 
focus of evaluation In local school districts. It has been suggested that 
administrators who wish to ensure that the evaluations focus on their dis- 
trict's needsbc prepared to respond to the following types of questions: 

1. What does the word "evaluation" mean as applied to this 
particular situation? 

2. What is the purpose of this evaluation? 

3. To what audiences will the results be reported? What 
are their particular information needs? 

4. What specific areas of the school program are to be 
eval uated? 

5. What types of evaluative information are needed to 
evaluate the program? 
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III. PROVIDING AN ORGANIZATION AND SUPPORT FOR EVALUATION 



One of the most important ways that the chief administrator of a 
hool district can contribute to the effectiveness of the evaluations 
d influence the staff's attitude toward evaluation is to provide for 
^ equate organization and support of evaluation. Failure to provide 
' ^is support most often results in limited use of evaluation, an over- 
jrdening of staff who already have many other established demands on 
leir time, conduct of evaluation by staff members with little or no 
^aining for the job, hastily designed and conducted evaluations, ques- 
■ onable results, and generally a bad taste in everyone^s mouth concerning 
' ne value of evaluation to themselves and the school district's programs/ 
n essence, if administrators do not provide the organization and support 
^ or evaluation, it is perhaps best to leave it undone. 

The factors that should be considered by administrators who are 
eviewing their present evaluation organization or who are beginning to 
establish an evaluation capability in their district include: (1) the 
type of organization to establish; (2) the placement of the organization 
in the administrative structure of the district; (3) the policies and 
procedures guiding the conduct of evaluation; (4) staffing; and (5) resources 
such as materials, facilities and financial support* 

Types of Organization for Evaluation 
in Local School Districts 

There are several types of organizational structures that can be 
utilized for evaluation in a school district. The best type of organization 
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Is the one which best meets the district -s evaluation needs. Prior to 
the development of an organization for evaluation, administrators should 
obtain some preliminary information regarding the purpose they wish 
evaluation to serve* the extent of the evaluation services they wish to 
provide and any resources--personnel , materials, equipment , facilities, 
monies s etc, --that are currently available and can be obtained to support 
the evaluation effort. Then as they consider the types of organization 
that can be developed, they can use this information to develop the most 
appropriate organization for evaluation in their district. 

Any organization plan that is established needs to provide the 
district with services such as evaluation design, Instrunient selection and 
development, data collection, analysis and interpretation and reporting of 
evaluation results. Stufflebeam (1973) suggests four such organizational 
strategies that may be appropriate to local school districts: (1) in-house 
evaluation departments; (2) self-evaluation; (3) contracting with external 
evaluation agencies; and (4) evaluation consortia between several school 
districts. 

In-house Evaluation Departments 

In this organizational strategy a position and/or department of 
evaluation is created within the district. This type of arrangement would 
be particularly beneficial to those districts who wish to utilize evalua- 
tion results for decision-making because of the proximity and constant 
avallabiHty of the evaluators to the rest of the school staff, 

A second benefit of this type of organization is the inclusion of 
personnel among the district staff who have evaluation training and skills. 
Technical evaluation responsibilities can be assumed by these personnel 
relieving this burden from busy staff members who may not have adequate 
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training for the job. The possession of evaluation skills in one or 
more positions also avails the district of staff members who can readily 
train others to conduct and use evaluations. 

A problem with this arrangement 1s that there are not enough trained 
evaluators to staff each school district in the country. In addition, for 
some districts, the development of an evaluation position or department of 

evaluation is too costly. 

Another problem arises when the primary purpose for evaluation in the 
district is to report accomplishments to external audiences. The proximity 
of the evaluators may, 1n reality or in the minds of some audiences, 
decrease the objectivity and credibility of the evaluations. To combat 
this criticism, districts who have a position or department of evaluation 
may have to secure the services of external experts in evaluation to review 
the work of the district's evaluator(s) and give credence to the results 
they produce. 

Sel f-Evaluation 

A second organizational possibility is self-evaluation in which 
evaluation is conducted by persons in existing positions in the district. 
If the administrator is fortunate enough to have existing staff members 
who have sufficient skills in evaluation design, measurement, and data 
analysis and interpretation, most, if not all, of the evaluation tasks 
can be conducted by the school district staff with little or no outside 
help. If these skills are not available within the district, additional 
external assistance will need to be obtained. 

Several kinds of assistance in this area can be provided to a district. 

Most cotmion are the contractual services of individuals or agencies who 

can provide specialized evaluation services. Similar consultant services 

can usually be provided at no cost by the state education agency, A 
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second kind of assistance could be provided in tested materials which 
guide groups through particular kinds of evaluation activities. Unfor- 
tunately, packages of this type are not common and are generally difficult 
4 

to locate. 

The importance of obtaining outside assistance when these skills are 
not available in the district cannot be overemphasized. The design and 
conduct of adequate and accurate evaluations requires knowledge of scien- 
tific inquiry, measurements statistics and interpretation of results. The 
risks to the quality and usefulness of evaluations of failure to include 
this kind of expertise 1n the design of evaluations are far greater than 
the expense of obtaining this assistance* 

' ,r EXAMPLE 9 

In a school district in the East the decision was made to 
establish a self-evaluation system In conjunction with the 
district's planning efforts* The superintendent and two or 
three of his key administrative staff members had rather 
sophisticated evaluation and research skills. The superin- 
tendent stated, "I feel that we have 80% of the technical 
knowledge and skills that we need to design and carry out 
evaluations in the district. For the remaining 20% we 
hire outside specialists and agencies who have materials * 
services and experience in the areas that we need assistance." 

There are several benefits to a self=evaluation system. The major 
benefit is in the potential of Incorporating evaluation as a responsibility 
and helpmate of all staff members. Initial costs to train the staff and to 
design ongoing evaluation procedures may be high* but when these activities 
are accomplished, this type of organization may be cheaper to maintain. 
Finally* a self=evaluat1on forces staff participation 1n decisions regard- 
ing evaluation and the conduct of the activities themselves. 



One such package is a document on Curriculum Evaluation developed by 
Research for Better Schools, It is currently being revised and will be 
available in November, 1975, 
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The major problem associated with this organiiational scheme has 
already been discussed— developing adequate plans and procedures for 
evaluation when staff members do not have the necessary training and skills. 
The concerned administrator can alleviate this problem by effectively 
utilizing external services and incorporating 1n this use In-service 
evaluation training for the staff. Secondly, a self evaluation system is 
more difficult to manage and control. Coordination of evaluation efforts, 
conimuni cation of results among staff and constituents and obtaining uniform, 
high quality evaluations are especially difficult. Finally, it is extremely 
difficult to control the biases of persons who are evaluating the programs 
they also direct. These biases decrease the objectivity and credibility 
of results that are obtained. 

Contracting with External Evaluators or Evaluation Agencies 

Local school districts may choose to contract with external agencies 
to provide the technical evaluation services that they need. This Is a 
familiar. model for school districts who utilize the services of test com- 
panies to provide instrumentation, score tests, analyze the results and 
provide reports of the results for a variety of levels within the district. 

Evaluation specialists and agencies can provide most of the evaluation 
services that a school district requires. They can assist in the develop- 
ment of a comprehensive evaluation system for the district, provide tech- 
nical services such as evaluation design, instrument development, data 
analysis and report development, and provide in-service training for the 
district staff. 

There are several benefits to this type of arrangement. Teams of 
highly skilled and experienced evaluators that would otherwise be unavail- 
able to the district can be utilized. If this is the case, administrators 



can expect to receive carefuny designed evaluations and accurate results. 
Through direct contracted services or by observation, staff members can 
obtain valuable in-service training. This kind of assistance, particularly 
if the individual or agency conducts the total evaluation and Is known to 
produce high quality work, is usually highly credible to external agencies 
and the public* 

The primary Impediment to the use of external agencies is the often 
high cost of their services. Secondly, because they are external, these 
agencies may not have the opportunity to become as familiar with the dis- 
trict and its. particular evaluation needs as is desirable. This same fac- 
tor may also create some fear and resistance in staff members. Third, 
great care must be taken in selecting the agency or individual who is to 
provide the services. Unfortunately, some individuals and agencies promise 
more or better quality work than they deliver. A final limitation is 
that the district may come to depend solely on the external evaluators to 
assume the responsibility for evaluation and neglect its own respons1bil1= 
ties in this area. 

Participating in a Consortium for Evaluation with Other School D istricts 

If administrators wish to utilize the services of trained evaluators 
and technicians on a continuous basis but are unable to establish the 
kinds of services they desire within their district, they may wish to join 
with other districts to form a consortium which would provide these ser- 
vices. Districts can pool their resources and establish policies to guide 
the provision of services such as conducting system-wide evaluationSi 
technical services, consultant services in proposal development and special 
evaluation problems , and in-service training of staff. 
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EXAMPLE 10 



Perhaps the most well -known of the evaluation consortia was 
the EPIC Evaluation Center located in Tuscon, Arizona. 
The Center was initiated through an ESEA Title III pro- 
ject and was designed to provide evaluation assistance 
to fourteen school districts in the state. The Center 
had two major service areas* (1) the evaluation services 
division and (2) the field services division. The evalua- 
tion services area was designed to provide assistance 
to school districts through- (1) the divelopment of 
evaluation designs; (2) test construction j (3) data pro- 
cessing; and (3) information storage and retrieval. The 
field services had the responsibility of initiating self- 
evaluation within the school districts by providing con- 
sultant services, providing Information regarding innovative 
practices and providing in-service workshops. The Center 
was administered by Its own director and staff under the 
direction of a board of directors which included the super- 
intendents of the participating school districts and 
operated according to policies approved by this Board 
for its operation. 

(Hammond ^ no date) 
The advantage of a consortium is that it provides needed evaluation 
services by skilled consultants who are in continual contact with the 
district at usually a lower cost. The limitations are that this service 
must be shared and the general difficulties of developing, implementing 
and maintaining services to districts with divergent interests and needs. 

The type of organizational framework that a school district adopts 
for evaluation may not be as clearly defined as those that have been pre- 
sented here. Administrators could conceivably utilize the benefits of 
each of these types of organizations simultaneously. The importance of 
reviewing these organizational types is to acquaint the reader with the 
strengths and weaknesses of each type so that decisions regarding the type 
of evaluation organization to be Installed can reflect the most important 
needs of the district. 
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The Placement of Evaluation in the Ofg i n ization 



The place or position that evaluation occupies In school district 
operation can have an important effect on the credibility of results^ 
pervasiveness of evaluation serviceSt and degree to which evaluation 
needs are met. Credibility of evaluation results 1s most often perceived 
to be a result of the degree of objectivity with which a program is 
evaluated. Objectivity is quite often perceived to be a matter of 
distance. The closer the evaluator is to the programs the less likely 
the results will be completely objective. It is for this reason that it 
is suggested that evaluators be organizationally separated from the 
control and influence of program personnel. For some districts this is 
achieved by placing the evaluator or evaluators in a position separate 
from all programs and reporting directly to the superintendent* This 
same result is achieved in the situation where external consultants 
reporting directly to the administrator are used. In situations where 
this type of arrangement is not possible^ credibility and objectivity 
can be enhanced by establishing a periodic review or evaluation of 
evaluation activities and services by a group external to the evaluation 
and program. 

One of the most difficult to achieve aspects of an evaluation 
system is a pervasiveness of evaluation activities and services to all 
of the representative groups in the district who need and desire evalua- 
tion help. Because evaluation activities are often limited, it is 
common to find that results and services are somewhat isolated from 
the ongoing operations and problems of the district* This is particularly 
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true in districts where the primary purpose of evaluation is to meet 
external requiriments or carry out routine, established data gathering 
activitias. In order to provide evaluation services to all levels of 
district personnel, e,g. teachers, principals and central office 
staff, evaluators should be located organizationally so that they can 
have easy and iniTiediate access to personnel at all levels and so that 
district personnel can have equally easy access to them. 

If the purpose of evaluation in a district is to provide Information 
for decision making, evaluators must be placed in a position that will 
allow them to be aware of the decision making and, therefore, evaluation 
needs of the district. This usually means that the evaluators need to be 
placed in a position close to key decision maker(s) in the district* 
The ability to provide needed and appropriate evaluations then becomes 
largely dependent upon the leadership capabilities and interests in 
evaluation of the administrator under which It Is placed. 

EXAMPLE 11 

Recently a prearranged call was made to a local school district 
superintendent in another state to discuss h1s evaluation program. 
His response to the question, "What kinds of evaluation activities 
are conducted in your district?'' was "Well, actually, we don't do 
much evaluation here, but I guess you could talk to our Director 
of Research to find out what they are doing in that department." 
Conversations with the Director of Research revealed that the dis- 
trict had a Department of Research with 20 staff members that was 
conducting all of the distrlct-^wide testing and federal program 
evaluations* Most of the activities they conducted, however, were 
required by law, reported to the proper audiences and seldom used 
within the district* The Department and its Director appeared to 
be fairly well isolated from the administration of the district 
and were scheduled for cuts in staff and programs for the upcoming 
year. 
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In contrast, a visit was made to a dlstflct where the respon- 
slblilty for evaluation was shared by two researchers under 
the supervision of one of the district's assistant superin- 
tendents who had, with the approval of the superintendent # 
established the small research and evaluation unit. They 
began their evaluation program by conducting studies that had 
immediate applicability to the district as a whole and sought 
early in their development to provide evaluative services to 
principals and teachers. Both representatives of the admin-^ 
istration and principals in candid interviews discussed the 
way 1n which these studies had been of value to theni and their 
plans for requesting similar services in the future* 

As was the case in selecting the type of organization to Installs the 

location of evaluation depends on the particular needs and climate of the 

district. Location is very important to the effectiveness of evaluation, 

however, because the placement of evaluation responsibilities can, in 

many cases, guarantee vlsibnity and use or isolation and even-^ 

tual abandonment of evaluation in the district. 



Poli cies and Procedures 



A great deal of misunderstanding and confusion can be avoided if the 
district has a set of policies and procedures for the evaluation activities 
that 1t conducts. These policies and procedures might include statements 
regarding the following: 

1, The meaning and purpose of evaluation 

2, The organization for evaluation including lines of authority 
and communication and the relationship of evaluation to other 
positions, departments and programs. 

3. The roles of staff and outside consultants 

4. Job descriptions of participants in evaluation 
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5* Procedural policies 



a, use of outside consultants 

b, evaluation design 

c, instrumentation 
analysis 

e, reporting audiences and formats 
6. Ethical considerations 

a. use of human subjects 

b. release of Individual results 

7* Provisions of support 

a, released time for personnel 

b. materials 

c. facilities 

d, sources of financial support 



EXAMPLE IB 

In order to obtain some uniformity in the testing and 
evaluat1on_jctivit1es of the three subdistricts In the 
decentralized Portland Public School system^ policies 
were developed with respect to the city-wide testing 
program and cross-subdi strict program evaluations. 
Areas included in these policy statements Includadi 

I, City-wide testing program 

a. types of tests and grades to be included 

b. setting standards of performances 

c. sampling 

d. maintaining and use of item bank 

IL Cross-subdi strict program evaluations 

a* method of Identifying common goals 
across subdistricts 

b, method of selecting instruments 

c. type of instrumentation - 

d, type of cost measure 

e, type of score 

f. type of evaluation and balance 
(formativep summative) 

g. format of report 

(Doherty, 1975) 
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Setting down these policies would not only provide a common basis for 
conducting evaluations within the system but would also provide some help- 
ful guidelines for external evaluators who are not familiar with the 
district* 

Staffing 

The key to the quality of evaluation efforts is largely a function 
of the adequacy of the staff that designs and carries out the necessary 
activities, What^ then are the skills that an administrator should seek 
in either internal or external evaluation staff members to ensure high 
quality evaluations? 

Michael Scrlven (no date), one of the, country *s foremost evaluators* 

says that evaluators need every skill known to man. Although we wish to 

think that he was being somewhat facetious ^ there are many evaluators who 

would be inclined to agree. Blaine Worthen (1975)5 another leading 

evaluator, chose to take a somewhat more narrow view when he defined 

25 general research and evaluation tasks and related competencies. 

Although specific competencies were listed for most of these evaluation 

tasks 5 space does not permit their reproduction. From these tasks, 

however, the reader may be able to obtain a general Idea of the skills 

that are needed to design and conduct evaluations. 

1, Obtaining information about an area to be 
researched or a phenomenon to be evaluated. 

Z. Drawing Implications from results of prior 
research and practice. 

3* ConceptuallEing the research problem or 
defining the object of evaluation* 
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4, Selecting an appropriate Inquiry strategy 
for addressing the research or evaluation 
problem, 

5, Formulating hypotheses or questions to be 
answered by the study. 

5. Specifying data or evidence necessary for a 
rigorous test of the hypothesis or an une- 
quivocal answer to the research or evaluation 
question. 

7, Selecting appropriate research and evaluation 
designs to collect data to test the hypothesis 
or answer the question, 

S, Identifying the population to which results 
should be generalized, and selecting a sample 
of the population. 

9. Applying the research or evaluation design 
and recogniEing or controlling threats to 
validity- 

10. Identifying at appropriate levels of gener- 
ality the goals of the program to be 
evaluated. 

11. Assessing the value and feasibility of 
program goals. 

12. Identifying standards or norms for 
Judging worth of the phenonmenon to 
be evaluated* 

13. Translating broad objectives Into specific 
(measurable) objectives. 

14. Identifying classes of variables for measure- 
ment. 

15. Selecting or developing techniques of 
measurement. 

16* Assessing the validity of measurement 
techniques. 
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17. Using appropriate methods to conect data 
(tests, InterviewSi unobtrusive measures, 
etc.) 

18* Monitoring the program to detect deviations 
from design or specified procedures, 

19. Choosing and employing appropriate techniques 
of design or specified procedures. 

20. Using electronic computers and computer- 
related equipment. 

21. Interpreting and drawing appropriate con- 
clusions from data analysis, 

22. Reporting research and evaluation findings 
and implications. 

23. Making recommendations as a result of the 
evaluation. 



24. Providing Irmediate feedback on program perfor- 
mance for use 1n decisions about program modifi- 
cation, 

25, Obtaining and managing resources (material and 
human) necessary to conduct the research or 
evaluation study. 

Lists such as these can be helpful to administrators as they seek to 
select a staff to conduct the evaluations for the district* They can be 
used as a guide to determine the evaluation skills that are already avail- 
able to the district through Its existing staff and simultaneously used 
to identify those skills^ that are yet needed. 

Personnel possessing needed, necessary skills can be obtained In a 
variety of ways. They can be recruited and hired as members of the district 
staff. They can be hired on a more temporary basis as consultants. If 
the district Is near a university, graduate students who have required Intern 
ship experiences may be requested. Existing staff members who are interested 
1n this area may be trained to conduct the necessary tasks. This is a 
particularly effective strategy for obtaining evaluation persdrtnel who are 
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familiar with thediitrict and are lore inclinid to r^efiiiin, li i s also 
an effective way of impr-oviTig the o verall e^al uation c apabi litiis o^" the 
district. 

In addition to th e technical s Ml Is re^ui redi an additional character- 
istic that is most desirable for esraluatlon staff members stiould also be 
considered--interface skins, lt\ p^rder for evaluatiori to be effectively 
utnizad 1n thedisii^ict there must be sati sfying aweriues ttiro ugh wN 
coiTiTiunlcati on betwe^h evaluators hni other 4taff tenibers car b< transtnitted^ 
As Henry M. Brickel (1975) stated in refer© Tice to ep^ternal ^valuaticns, 
"(Eval uators must have) the abil \ty to bite the hand that feed s you 
while seefnirg to be only licking it." E^ai uators wlio cannot effectively 
translate the resul Is of tlieir studies to 1 es& techrlcally cri anted 
audiences or v/ho la<k the tolerance and patience to ef fecti ^el y coimiiunicate 
their results decrease drastical ly the Ljill ity of ttiei r ^indirtgs^ 

— An Important a spect W siaf^fing is Hhe cantinu^us s 

of both the ev^al uators ard thfi other personnel v/Kio utflise the results . 
Educational eval u^t ion is a relatively' n«w field and, as a result, rew 
techniques and conceptualizations are conti iiuall^ b elng introduced. To 
maintain a quality staff ? administrators should encourage the develcpment 
of continuous and multi-level staff development act Ivltlfis in e\falijflti on . 

As for any other program conducted in the districts resources such as 
time I perSonneU niaterlals^ equipment i facilltieSi and dell ars should be 
allocated to evaliiation based on the %pe and scope of ttie actiyitles that 
are to be conducted. While this may seen) to tse an obwioiJs statefnenti it 
is coinmon to find that aclequaie provisions for e^al ution are not found In 

ERiC 55 



schcol district t^udgets and operational plans. Wl'thcut adeq uate resources, 
evaluation activities cinnoe b i expected to be ef^ec^iv^eli/ 1 nipleiTiented, 

How much of a di5lrict''s iudget s hould be dev^ot^d to evaluation? In 
Indtistry figures frm to 2 51 are coftnon . In education figures of II 
are extremely iiiph and as entr^niely uncominon* The answer to this question, 
of coune, depends on ihe activities that are tto be conducted. Extensive 
avaluatlon effcrts reqaire 1 arse budgets and opera^tionil platis while 
modest efforts re-qulre l^ss,. The innportant thHng for an administrator to 
consider is the p^rovision of a budget ihat v/IIT ge:t th& necessary job done. 

In a recent study of evalijatiori units In large districts 
in Texas the la rgest percentage of the b udget to spent 
for evaluation M In the D^ailas School Sjstein- Actual 
dollar expenditaires per pupil v/ere $6.20 , 32. 06^ $1>54, 
$1.4S, $]J9^ $ .15 and $.1 2. 

(Jackowskfl i M cMama ra l £01 6, 1975) 

Ju^%ry 

In t his sect ion atternpts hav* been niade to br 1ef ly di scuss those 

aspects 0 f organisation v^Siic h have t^he greatest imfpact cn the evaluation 

actf vi iie s conducieci Ir^ a school district, Adnlni ^tratcrs who i^ish to 

anaTyze their present or proposed organization for ev<alaiation might benefit 

■Trorn respcnding to these que stions: 

K Is the organi2ation.al structure such that It meets the 
distrlct^'S needs foT eyaluation? 

2^ 3s the locatio»fi of isvalustlcn wiihf n the structure appropriate? 

3^ Ave there established policies ard pro^cedures for evaltiation? 

4. firm the needed steiUs and afciT i ties &v Ident in th^ current staff? 
J*.^rndtj hay< adequate plans been dev&l oped to sectire tJiese skills? 

5. P^TB the alHca ted resources adequate t o accornplisSi the needed tasks? 
3f fioti can additional resources be obtained? If n^t, have activi- 
ties been reduced to refTeci available resources? 
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IV, COMMUNICATING EVALUATION RESULTS 

One of the most effective ways to achieye an acceptance of evaluation 
and encourage its use is to Implenient a carefully dsyeloped plan for commu- , 
nicating results to targeted audiences and obtaifiing their assistance 
through feedback. To date, however, this aspect of the evaluative process 
has been inadequately addressed ^ accounting, in some ifleasure^ for the 
limited utility of evaluation in school districts. Administrators interested 
in devel oping effective evaluation prografns in their schools need to be 
aware of the basic considerations associated with effective comiunication 
and to Install procedures into their operation whicti would guarantee an 
adequate program for conmuni eating evaluation results to potential users. 

In order to maximize the use of evaluation results, plans for the 
comiiuni cation of findings should be made at the same time that the evalu- 
ations themselves are designed. When this is done the schocl district 
staff can be prepared to develop timely and appropriate reports and the 
recipients of the results can be prepared to receive the results. 

In order to establish an effective system for conmiunlcating the 
results of evaluation administrators need to: (1) see that mechanisms for 
planning and conducting the communication program are established; (2) be- 
come familiar with the district's particiJlar audiences for evaluation 
results; (3) see that accurate and appropriate reporting procedures are 
utilized; and (4) establish channels and procedures to provide feedback 
from various groups regarding the results and to evaluate the effectiveness 
of the convnunication system Itself, 
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These are^ of course, types of acti\/it1is which characterize good 
public relations and communications programs in school districts* For 
those administrators that have such programs In their districts the 
following discussion will provide some suggestions for the inclusion of 
evaluative inforniatiori 1n their existing coimnunlcation prograni, For those 
administrators who do not have such established programs the following 
discussion is designed to acquaint thefn with both basic considerations 
for contnunicating information and specific suggestions related to the 
release of evaluative information. 

Providing the Framework for the Developnient of a 
Program "for the Re leas e of "EvaTuati on "Results 

Prior to specific planning for the release of any evaluatlQn results^ 
administrators need^ to establish the framework and mechanisms by which 
communication of evalLratfon resul ts wil l be conducted. Suggestfons such 
as the fonovjing can provide guidelines to those vfho will liltimately be 
responsible for carrying out communlcatiDn activities and providle a basts 
for judging the adequacy and effectiveness of the resulting cominuni cation 
effort. 

Determin ing the Pur pQse(s) for Communicating Evaluation Results 

Determining the purpose(s) of conmunication provides some direction 
to planning activities and provides a framework for evaluating the effec- 
tiveness of the effort. Bettinghaus and Mm er (1 973) suggest that there 
are four general categories in which most coninuni cation objectiv^as can be 
grouped. These are: (a) increasing awareness of the progrann; (b) changing 
attitudes toward the program; (c) achievi ng compliance with required tasks; 
and (d) obtaining supportive behaviors for the program. 



Froni these basic categories specific objectives for comnunlcating the 
evaluation results can be developed. Because the purpose for communicatirig 
evaluation results may differ for the various audiences to whom they v/m 
be reported administrators may wish to specify those major purposes which 
win ue directed to each of the major target audiences* 

EXAMPLE 14 

Montgomery County^ Maryland identified the following as 
specific purposes for their annual school progress reports: 

U Achieve greater parent knowledge and understanding 
of the needs and strengths of the school 

2. Achieve parent identification with and conmltnient 
to the objectives of the school 

3. Indicate how school programs are designed to meet^ 
the needs of students^ maximum use is made of avail- 
able resources^ and the need for additional resources 
is substantiated by reliable data 

4. Encourage a high level of staff and parent Involvement 
in school planning and support for the way the school 
uses available resources 

5. Be an integral part of the process through which all 
available data are reviewed to establish school objec- 
tives^ indicators of their attalnmentt and follow-up 
procedures 

6. Involve school staff in gathering and Interpreting 
available data on the community, students, and school 
resources to set school objectives 

7. Involve representative parents and students in a review 
of available data and the school objectives established 
by school staff 

8. Communicate school objectives ^ priority for action 
based on these objectives, and elicit support and 
reaction froin parents, area^ and county staff* 

(Elseroad, 1974) 
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Establishing Pol icies and Procedures to Guide CQmmurii cation Programs 

When the objectives for coinniunicating evaluatloii results have been 
determined, administrators should consider the development of policies 
and procedures i^hich would ensure the achievernent of these objectives. 
Policies for coimTiunicating evaluation results are particularly^ important 
because they provide a structure and unifomilty to conmijnicatlon efforts 
that ease the implementation of the prograni for the staff and protect the 
district from some of the destructive criticisms which may arise. 

Policies for the release of evaluation results may include: (1) the 
Identification of types of results v/hich cannot be released ^ e.g, those 
results which violate the rights of individuals; (2) those results which 
must regularly be released to specific audiences; ard (3) the channels 
through which evaluation results are to be released. Procedures and/or 
guidelines for communication may be developed to assist staff in developing 
an awareness of the needs of various audiences, identifying results to be 
reportedj developing cofninuni cation messages utilizing various media, 
gathering feedback and evaltiatlng the comnunlcatlon program* 

Establishing or Utilizing an Advisory Conmlttee for Evaluation 

Administrators may find it beneficial to include in the organization 
for communication 3 an existing or specially created advisory council, 
composed of representatives of the major groups Involved and Interested 
In the school system. Such a council might consist of representatives of 
parents^ students^ teachers, principals, central office staff, the board 
of educations rnajor community groups^ interested taxpayers, etc* The 
major purposes of such a council might be to provide input Into the selec* 
tlon of areas to be evaluated, to serve as a preliniinary sounding board to 
the results which are to be released, and to serve as one vehicle for the 
transmission of results to the public at large, 
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Identifying a nd Assi gning Responsibi li ties for Coniniunlcatioa 

A great deal can be done to ease the implementation of a communication 
program 1f the specific functions required to implement the program are 
defined and assigned to appropriate staff and advisory personnel. These 
functions may be developed in relation to the categories of (1) planning 
the comnunlcation prograni; (2) encoding or phrasing the messages in an 
understandable format for the Intended audiences; (3) transmitting the 
developed Tuessages to Intended audierices; and (4) gathering feedback con- 
cerning the messages v/h1ch were communicated (Simonp no date). Clearly 
when all needed activities are idertlfied and assigned to personnel, there 
is much less risk of Important activities being overlooked and of staff 
misunderstanding concerning their responsibni ties for the program. 

Developing a Schedule for the Release of Evaluation Results 

As administrators and staff assume responsibilities for reporting 
they should also establish a schedule for releasing the results. The 
timing of the release of results has a great deal of impact on the utility 
of evaluation and is, therefore, a wry important aspect of the total 
evaluation effort. 

Scheduling the developing of communications regarding evaluation 
results should take Into consideration (1) the time at which evaluation 
results will be avanable; (2) the time at which decisions must be made 
regarding the area to be evaluated; and (3) the tirnes at which target 
audiences will be available to receive communications regarding evaluation 
results. 

Additional consideration shouldl be given to preparing interim coimuni- 
cations regarding the results so that targeted audiences can have sufficient 
background to receive and use the results and so that they will be prepared 
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for the possibtlity of favorable or unfavorable resuUs. Sufficient 
lead time shouW be provided to inform those who would be asked about the 
resuUs time to acquaint themselves thoroughly with the findings. At 
the same time care should be taken to release evaluation results as soon 
as it is feasible to do so in order to prevent "leaks** of partial infor- 
mation which can be misinterpreted. 

Developing an Avjar eness of the Potential 
Recipients of Evaluation Results 

People most often respond to Information they receive if they are 
interested in it and can understand it. The first step in conducting the 
work necessary to carry out an effective communication program must be, 
therefore^ to Identify and become familiar with persons who will be 
potential receivers of evaluation results and their particular information 
interests and needs. Then, and only then, can district personnel begin 
to design and develop techniques for communicating results to these 
audiences which will be effectively received and used. 

Identify ing the A udiences for Evaluation Jnformation 

In the sicond section of this paper suggestions were given for 
identifying evaluation audiences and their needs. In addition to this 
information it is helpful for those who are developing communication 
messages to know the audiences' level of knowledge and attitude toward the 
area in which the evaluation Is conducted. 

The release of evaluation results is often characterized by the 
inclusion of new programs , terms and symbols which are unfamiliar to many 
audiences. It is, therefore ^ helpful to know the level of both knowledge 
and/or misinformation that various audiences have concerning areas that 
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might be discussed in an evaluation conmuni cation* This inforniation can 
assist communication developers in determining the exterit of background 
infoifination and pre-result comfnunication that is necessiiry to create coniTion 
understanding of the area being reported among the district and its 
audiences. 

EXAMPLE IS 

In a research study conducted 1n California 13 parents from 
the middle to lower economic class were asked to read 106 goal 
statements for elementary education. The parents identified 
1,265 words and phrases that they did not understand. The 
study pointed out the need to communicate 1n tunderstandable 
language and to make special efforts to explain new concepts 
to the school's publics. 

(Barnes, 1972) 

Finallys it is extremely beneficial to Identify th'S ;t1tudes which 
various audiences have regarding the total school program and the specific 
areas that may be discussed in evaluation communications , Thosewhohave 
negative attitudes toward the schools, particularly if they are in a 
position to jeopardize a program, need to be identified so that efforts 
can be made to attempt to change their attitudes or neutralize their 
effects. Those who have positive attitudes can provide iidmlnistrators 
with needed support for their efforts. 

EXAl^LE 1^ 

Educators in Pennsylvania received a clear indication of 
the opinions of some audiences concerning thair evaluation 
efforts when the following statement appeared In the 
newspaper: 

...a group of parents and educators, and the American Civil 
Liberties Union (ACLU)* **call attitude testing an invasion 
of privacy, damaging to children who are sensitive about 
revealing their Innermost thoughts. ("Student Attitude 
Called Privacy Invasion," 1975) 



Identifying the Forms of Coirniun icatl Which 
Are Best Received by Each Audience 

Everyone does not read the newspaper , watch television or attend 

PTA meetings. It is important In designing communications to select a 

technique for communication which will most effectively reach the greatest 

number of the specified audience. Attention needs to be givenj thereforei 

to the identification of those approaches which are most effective for 

specific groups, 

EXAMPLE 1? 

In a survey conducted as part of a community-education 
project conducted 1n a rural North Carolina school 
district the following results were obtained: 

The most effective method of communicating (to the 
target audience) about project activities were (1) a 
school newspaper entitled ''The Bond Community Enter- 
prise;" (2) church announcements ^ and (3) visits to 
homes by ESEA Title III home=school counselors. The 
least effective methods were by radio and newspaper, 

(Bazemore^ 1973) 

In summary, the effectiveness and appropriateness of communication 
efforts can be greatly improved if district personnel can identify the 
the following concerning the audiences: 

Areas Level 

Audience/Priority/ of / of /Attitude/Best Method of Communication 
Interest Knowledge 

Developing the Communication 

As can be gathered from the preceding discussions a great deal of 
work and thought must be given to a communication program before the 
first word of a report or message Is written. With this background, 
however, the task of developing communication is far more clear and less 
subject to error. 

6 1 
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The development of a specific communication system for evaluation 
results requires: (1) an identification of the evaluation Information 
to be reported to each audiences (2) the development of messages that 
meet the needs of each targeted audience; (3) tailoring the communication 
to the media through which it will be channeled; and (4) establishing 
channels through which comnunlcations and feedback will flow* 

identifying Evaluation Information to be Reported 

The information that Is to ^oe reported to each audience can be 
determined by comparing the kinds of information that will be made avail- 
able as a result of the evaluation to the identified Information needs of 
each audience. If this particular activity Is conducted at the same tirne 
as the evaluation itself is designed^ gaps in kinds of information needed 
can be identified. Procedures for gathering this needed Information cm 

then be incorporated into the evaluation desiga. 

As the evaluation information and needs of targeted audiences are 
compiled, it should be possible to detect similarities in the types of 
evaluation Information available and audience needs. When these similar- 
ities occur, it then becomes possible to utilize a single comnunication 
for several audiences. 

At the completion of this activity district personnel should have a 
fairly well defined plan for the number of communications that are needed, 
the audiences to whom each of these communications should be addressed and 
the Information that should be included In each communication. 

EXAMPLE 18 

PRIMES, an ESEA Title III project designed to assist 
schools 1n Columbus^ Ohio, to fulfill mandated evaluation 
requirements^ had as one of its activities the identifi- 
cation of audiences s informations dissemination means and 
schedule for reporting information concerning the project. 

59 

63 



Charts depicting their decisions for the total 
project and activities at the building level are 
presented on the following two pages* 

(Project PRIMES, no date) 

Deyel oplng the Hessages 

It 1s at the point of developing the comniunlca ^on that particular 
attention should be given to the nature of evaluation results, the par- 
ticuiar audience to whom the results will be reported and the media through 

which it will be transmitted. Some particular suggestions for reporting 

5 

evaluation results are: 

1, Begin with technically valid and accurate information so that 
a truthful account of district accompli shments can be reported; 

2* If the audience does not have a great deal of technical 
expertise^ summarize detailed technical results in clears 
nontechnical language. Save the technical Information for 
special or required reports; 

3, Utilize a more journalistic rather than technical writing 
style; ... _ , . 

4, Avoid jargon; use understandable words^ familiar language; 

5, Keep conmuni cations brief; 

S, Set a tone in the communication that is objective and 
affirmative; 

7, Provide sufficient background information to explain the 
results* implications of the results for future actions 
taken by the system; 

3. Sum up what the results mean; when possible provide com- 
parative and interpretive information; and 

9. Encourage realistic expectations. 

If the results are bad, Hawes (197Z) suggest the following: 
1. If they are better than before^ say so; 

2* Identify clearly future actions that the district plans to take 
on the basis of the results, 

3, Identify increased support that is needed to improve perfornriftnce, 

4« Avoid implicating individuals for the failure. 

^Some of the suggestions were taken from Shepherd (1975), 
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PROJECT PRIMES DISSEMINATION CHART 



Audiences 


Dissemination Means 


FrequOTcy of 
Conwiunicatlon 


State Department 
of Education 


Newsletters 
Written Reports 
Memos 


Quarti^rl^ 
Quarterly 


Congressmen 
Wiley and Devlne 


Letters 
Newsletters 

Packets of Information 


Semi ^Anniual ly 
Quarterly 


Superintendent 
of Schools 


Abstracts of Reports - (2 pgs) 
Newsletters 

Oral Presentation at Comittee- 
of=the Whole Meetings 

Indirect Memo Correspondence 
through Asst. Superintendent 


Semi-Amnwlly 

Quarterly 

Annually 

Annually 
As Needled 


Cabinet M ^bers 


Abstracts of Reports (2 pages) 
iisws 1 uer > 

Oral Presentations at Committee- 

of-the-Wtole Meetings 
Slide-Tape Show 

Memos . . „ ^ ^ - - - 

Consultations iPerson to Person) 


Sem1-.Anfiiuiny 

Quartarly 

Annmany 

Annually 

As Hm^M_ 
_AsJ^e^dl<ed 


Curriculum 
Specialists 


Consultations (Person to Person) 
Newsletters 

Curriculum Reports and 

KeCDIIiilcirMM u 1 Uii3 


As Needed 
Quarteiriy 
Annuainy 


Elementary 
Principals 


Slide-Tape Show 
Newsletters 

Consultations (Person-Person) 
Brochure 

Triri4 1 1 7^fi ^rhnnl RpDort 

Information Packets 
System Newsletter 
Curri cul um Report 


AnriuaiHy 

Quarterly 

As Nee'ded(Min1mum 

Twice a ilear) 

Annual ly 

As Ntedtd (Twice Sr.] 
AnrnMany 

, Annually _ 


Teachers 


Slide-Tape Show(Updated Yearly) 

Newsletters 

Brochure 

Consul tatlQns(Person to Person) 
Curriculum Report 


Annyally 
Quartwly 

As H^iadiad 
Annually 
Annual 1/ 


Parents 


Surveys 

Cormittet Heetings 
Newspaper Articles 
Slide-Tape Show 
Television Presentations 
Radio ProgT^ams 
Brochure 67 


Annuail ly 

2-3 l\mm Yrly, 

2-3 Timas Yrly, 

Annwl ly 

Yearly 

Twicf Yaarly 
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EXAMPm (CONT'D) 
DISSEMINATION AT THE BUILDING LEVEL 



Content to be 
CommunlcaWd 



Area of scii^Ciol 
program to be 
evaluated 



Evaluation 
procedures 
ut1l ized 



Staff and 
Community 
-involvement 



Reporting 
Eval uatlon 
Results 



Follow-up 
Acti v1ty(s) 
to the 
Evaluation 



ConmiunlcatQr(s) 



-Building 
Evaluation 

Cornmittee 
-School Staff 



Building 
Evaluation 
Conni ttee 



Building 
Evaluation 
Comni ttee 



Bunding 
Evalution 
CofTmii ttee 



'School Staff 
^Principal 



-P^ rants 

^PRIMES Staff 

-Statf Dfpt. 
of Iducation 

-Curri ai^l urn 
ArM Special- 
ists 



^Parents 
^Stat®' Dept* 
of Education 
^PRIMtS Staff 
^SchQOil ..Staff 



'-School Staff 
^PRIMCS Staff 
-Sirvice! Clubs 
ana Oirpniza- 

-Local Citizens 



-Services Clubs 
-SchO!(>1 Commun- 
ity 

-PRIMES Staff 
■^Stata 0$pt. 
of Education 
-School Staff 
-Curriculum 



Dissemi nation 
Media 



-Newsletter 
-PTA Meeting 
-Memo 

^Standardized 
report form 



"Standardized 
report form 
-Memo 

-PTA Meeting 



-PTA Meeting 
-Staff Meeting 
-Standardized 
report forms 
-Local News- 
paper 

-School News- 

i etter 
-■PTA News-- 

1 etter 
-Service 

Meeting 



-Parents 
-Service Clubs 
-Schoial Comirun= 
ity 

-PRIMES Staff 
^State Dept. 

of Edijcetion 
-Curriculum 

Special Ists 



-Special Kept, 
-Newsletter 
-Transparency 
report 
-Grade card 
insert 
-PTA Meeting 
-Local News- 
paper 

-Educational 
Journals 



-Special Rprt* 

-Display 

-Slide-Tape 

presentation 
-Demonstration 
-PTA Meeting 
-Service Club 

Meeting 
-Newsletter 
-Local News- 

paper 



Impl ementation 
Time! Tne 



Early in School 
Year 

(Sept. to Oct,) 



Early 1n School 
Year 

(Oct, to Nov.) 



Mid-Fall 
(Nov. to Dec.) 



Winter 
(Jan. -Feb.) 



Spring 

(March to May) 
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In addition to t^e nature of <valuatiori fesults, conslderat ion should 
be given to the particular audienc* to wliom t*e resuHs will U reported- 
Communications should be developed wi th an un^erstand^ng of tlie relevance 
of the 1 nformati on to the sudlence and the au dience's techrilcal understarid-' 
1ng, political per-spe<t1 ves and attention span. In otfder to «n&ourage tw/o- 
way communication concernlrig the results, nes sages should include a pro- 
vision for feedback from tne audience to tlie distr-ict. 

In order to effectively utilize media, dis-trlct personnel nmst becoiiie 
aware of the my messages should be conStr»icted for the media that lias 
been selected. Telivisfon spots are clearly dlfferen t froni riewspaper 
articles both of wnich are considerably different irom a 25 pagewritien 
report. Message creators Tieed to taNe into consider'atlon ihe require <l 
journalistic styles f ortfiat ^ length, and particular strengtiis of each ned iunn 
before submitting a cominun icsti on to its use, 

||tabljsMna_Jhann«Ts^ 

It is importast to have es tabli she-^ cliarinels ihrough. ^h1ch CoPmunW 
cations may flow prior 40 its release. This not only esses the process of 
releasing results but also gives newspa permeni program cha irrnen ^ eic 
sufficient notice to scliedule the cannu nicafion arnong tinei r other activi- 
ties and provide any batfkgrotindor lead- in cofmiinicatlons prior to ihe >'elease 
of the final resul ts, This can be done by initial ly cofita.cting personnel 
through which conmunica'tions will be channeled, explaining the purpos-e 
of the evaluation and tfie coniiiuni cation of its results and receivl ng from 
them suggestiCo>s as to -the mcst appropriate use of theiT nied-iiri. 

It Is then Qqi.i-5*ly Important to nurture and malntaiwi a good workiiig 
relationship with theia perscns in order to piak* the ine'^lwiii a viable tool 
for coninunl cation and f-fiedback. Ihi s &a;i be ac<onipHsh<d by relsf. sinp 
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resulti at th^ scheduled tirae in a forfiiat app^rc3prf ate t€ the medium, being 
Gperis •available a nd trgth^ul and by being pripared to p rowide fonow-up 
f nforin-atl on if requested. 



Effective coHnmufiicat iom is a tvJO-my process, ^dntintstrators who 
raleas* evaluailon result^ ras^ be prepared to reciiwe and u"til Ize the 
feedback they Tecelva froff) targeted au^i^rices. An initial s'tep in this 
diirect iort v/ouH be to pro^icBe and pLibHd ie well-kno\Nn channels for feed- 
bacH. TKese may include parent-teacher conference s, specfally called 
hearings on therisultSi PTA* (neet ings, letter nrit ing o r personal contact 
with tile adrninist rator or h1 s or her s^a-ff, Then as feidbacS< Is received 
administrators ne'ed to utilize the restilts in prog rarr p larifil fig ^ designing 
new e\/aluations a^Tid Inipro^in^g the cotnimLini cation pr^ogram itiseTf- Publlca- 
tioh 0^ the use tihe district has made cf thefe*edback is a valuable way 
to enccurage addliional i*ipijt and ccpntnnuQus fgedb acN f^or fu"ture evalua- 
-Cion efforts. 

MtmT the results of m e^al uatiofi have been released afid feedback 
has be<n received ^ d]sttr1-ct personnel ne&d to snal jze thai r connniuni cation ^ 
prografiiand the knowledge tNey hai/e received. Thi i-i ©na lysis can be as 
formial as a deiai led ewal Mtion d^si gn to gather s pecif led t^pes of 
Information or as informal as s disciission of the program at a staff 
rneetins. TNe tar^flets of ^hi s snalys^is should be tOie concerns of the 
targeted audiences and the e ffect ive-ness of the coTOiun1<atiori strategy 
that was used, 

Time should fce taton to reassess the knowledgi6j concerns and interests 
O'f target-ficl audiemces to ensure ttiat info rnistion t&at will be used as a 
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basis for future communications efforts is up-to-date and accurate. 
This analysis should also provide an indication of sources of support 
for the schools and their evaluation efforts as well as sources and 
reasons for criticism. 

A second analysis should be conducted to determine the effectiveness 
of the communication and feedback strategies utilized* Questions such as 
the fonowlng might be investigated: 

1. Did the messages reach the intended audiences? 

2. Were the concerns of each audience addressed? 

3. To what extent were the objectives of the comunication met? 

4. Was sufficient ground-work and background {nForinati^n provided? 

5. Were the avenues for feedback effective? , Which wre rost 
heavi 1/ used? 

6. Were the operational plans for the prog?™ feasible and workable? 

7. Were the messages appropriate for the selected media? 

On the basis of this analysis ^ necessary modifications in the 
communications prograni should be installed to improve the communication 
of results for future evaluations. 

Summary 

As can be seen from this discussion^ the communication of evaluation 
results 1s a highl/ complex undertaking requiring a great deal of planning^ 
vork and specific skills in effectively relayiiig a fnessage through a 
variety of rnedia to different audiences with varying Interests and needs. 
Because effective cofTinijni cation 1s the key to utilization of evaluation 
results, 1t is critical to the total evaluation process. Administrators 
need to be aware of its contribution and utilize their resouirces to develop 
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a planned prograrR^to coimunicate their results to selected audiences and 
gather iniportant feedback to assist them In the improvement of the dis- 
trict's programs and receptiveness to its constituents. 



V* UTILrziNG THE RESULTS OF EVALUATION 



An mmluation is a waste af msoi^ma unte&B it l&ada to 
somm aation^ at Z&ast a aonsaCouB^ escpHGit deoision 
not to take any amtion^ 

Brlandscn (1973^ p, 2S) 
Educational evaluations are judged mny criteria such as the 
accuracj^ of the results and the appropriateness of the methods used* One 
of the m%t important criteria from the position of a school administrator 
iSf hov/everj the utility of -:he evaluations. 

Ensuring that results &re used in a school district is one of the 
rnost iniportant of the adrni n1 strator' s responsibilities for evaluation. 
It my also be one of their most difficult and frustrating tasks. As 
e)(perience v/lth testing prograiris has showiis the mere provision of evalua- 
tion results does not guarantee that tb^^ be use«3* There are other, 
factors s related to the evaluations themselvts and the environment In 
which thej^ are released, that have a considerable effect on the extent 
to vihlcli evaluaticni are used. Such factors include: 

Ths dsgrem ta whiah mvatuat'M i*&&uita addrmBB the 
speaifia n&eda of €ts mvlL^naes; 

Evaluation results vvlll be more readllj^ used if the persons who 
receive them can see a direct application of the results to their particu- 
lar needs, When evaluation results do not directly address their needs^ 
udrgeted audiences often have difficulty determining the implications of 
the results far their work. It is for this reasori that such emphasis is 
given iri the focusing and conmuni eating stages to determining the audiences 
for evaluation and their information needs. 
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The mt&ni? to whiah th& inimnd&d ub&P3 weim iyri^otwed 
in the evaluation proamsm and^ there fw&^ f&&t com- 
mit't&d to the effort and its t^esulte; 

Stuff! ebeam (1974, p. 25) stcttes , "A principle of educational change, 
is that unless persons who will need to support the change are Involved 
early, ineanlngfully and contlriiously in the the developmerit of an Inno- 
vation ^ th&y likely will not support the operation and use of the Innova- " 
t1on*" The same principle applies to the utilization of evaluation results, 
Involvememt, comnii tmentj cooperation, credibility and use are closely 
related suggesting that administrators wishing to max1iTi1ze the utilization 
of evaluation results earnestlj seek the involvwnt of targeted audiences 
in planning and/or conductiiig evaluations. 

3, mt&nt to wHioh the midienae p&ra&%V0B thm TmButtm 

to be true J 

It is an almost impossible task to have people use evaluation results 
that they perceive to be In error. It is for this reason that adminis- 
trators should continuany strive for excellence in their evaluation 
efforts. To convince audiences that evaluation risults are true ^ adminis- 
trators should coDTiunicate to- tlieir audiancen the ma thodol ogles used to 
collect evaluation data focusirig on those efforts that v/ere made to 
ensure objectivity, 

4, The extend to whiah mdisnaes pei^aeive the uss of 
evatuation x'e suits to be impoTtantj and 

The degree of importance that the district administration attaches 
to the use of evaluation results has a marked effect on the utnization 
of results by the staff and public. Administrators wishing to encourage 
the use of evaluation results in their districts need to convey the impor- 
tance of using evaluation results by providing support to utilizstion 
efforts. Staff in'-service and public meetings to explain the results 
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and provida suggestions for use and rtVaased tifne for staff to study the 
results and plan on the basis of the findings are but two ways that the 
importance of using evaluation results might be conveyed. 

S, 0i# emtent to wMoh the staff and pubtia are witling 
to aaaept dharigm. 

It is often the case that evaluation results indicate a nmi for 
change in the way things are currently and^ sometiTnes, traditionally 
being done. Educators and publics that are resistant to change will 
profaablj^ find very little use for these types of results. Administrators 
who actively engage In evaluation activities in their districts must be 
prepared to make changes and encourage their staff fnembers to alter 
their methods when evaluation rosults indicate that such a need exists* 
This is not an exhaustive list of factors related to utilization. 
It should^ however^ provide administrators with some thoughts concerning 
the factors that might affect the utiliMtlon of evaluation results in 
their particular districts and suggest some steps that might be taken to 
maximize the use of their evaluation efforts. 

Given that administrators i their staff and publ1(2s are receptive 
to using evaluation results^ for what kinds of actions can they be used? 
The answer to this question is dependent on the purposefor which the 
evaluation was conducted, the type of information that was provided, and 
the Imagination and creativity of the recipients of the results. Some 
assistance in determining how evaluation results should be used may be 
provided, hov^ever, by lists of examples like the follov/lng: 
Administration 

1. Support and defend administrative decisions 

2, Establish dlstrict-wide priorities for programs and 
Operations 

3* Allocate funds 
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4. Reduce wasteful use of resources, i.e. rnoneys 
personnel, materials, equipment^ facilities 

5. Determine pupil and/or public satisfaction/ 
dissatisfiction with schools 

6. Continue or terminate special and/or innovative 
pro^ficts and programs 

Curr icula and Instruction 

1. Identify rnethods and fnaterials which best meet 
student needs 

2. Choose curricula 

3. JdCTtlf^ needs 

4. Establish district-wide, school building and 
classroom goals and objectives 

5. iMotniitor student progress 

6. Monitor program development and progress 

PersQinn^l 

1, Identify in-service training needs 

2, Assign staff 

3, Grariit prorations ^ tenure, special supplements, etc. 
4* Terminate employment 

Public 

1, Htlp board of education account to students ^ 

parents and citizens 
2* Bring public needs and opinions to the board 

of aducation and administration 

3, Improve cofnniunlcatlon to publics 

4, rncfiaase the knowledge of publics concerning 
schools 

5, Encourage greater public Involvement in 
school affairs 

Sovern1r:ijj/ Funding Agencies 

1. Justify expenditures of funds 

2, Document effects of special programs 

Practicing administrators ^ reading a list such as this, can undoubtedly 
think of many more ways that evaluation results can be used in their dis- 
tricts. For rnpst, however, the transition from thinking to actual use of 
results win not be easy, The history of the use of evaluative type 
information in publUc schools 1s not a good one. Perhaps the most 
outstanding example is In the use of standardized test results. Most, if 
not all, childrein in toerican schools are given standardized tests, 
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Unfortunately I the results of these tests have seldom been used to plan 
instructional programs for the students. Administrators who wish for 
evaluation results to be used in their district may find that they must 
work particularly hard on this aspect of their evaluation program before 
results are used to a great extent. 



71 



VI. THE EVALUATOR'S ROLE 



The primary focus of this document has been those areas of 
. responsibility for evaluation which are directly within the realm of 
school district administration. These have included: (1) establislring 
a supportive climates (2) focusing evaluation on the needs of the district; 
(3) providing an organization and support for evaluation; (4) maintaining 
a system for communicating the resultsi and (5) utilliing avraluation 
results. CI early I these are not all of the roles and responstbil ities 
that inust be assumed to conduct evaluations in local school districts. 
Those responsibilities which are associated with the actual planning and 
implenientation are noticeably missing. 

The reason for this omission Is that designing and implementing 
eyaluation studies are not major respDnsibll ities of administrators but 
of evaluators and/or the persons in $chool districts who actuany do 
the evaluations* these are the activities that require the specialized 
skills of Inquiry, i,e, identifying evaluation questionSt designing plans 
to obtain necessary Informations Instrument selection and devtlopment^ 
and data analysis and interpretationi and are ^ therefore s best accomplished 
by persons specifically trained in these areas, 

A more complete picture of the entire evaluation process can be 
obtained by reviewing those major responsibilities of evaluators. These 
responsibilities incUide: (1) planning evaluations; (2) selscting or 
constructing instruments; (3) collecting data; (4) processing data; 
(5) analyzing and interpreting information; and (6) reporting evaluation 
findings, ^ 

These and many of the tasks given were taken from Owens (1968, pp, 74-75). 
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It 1s at the planning stage that evaluators can and should dr4w 
TOSt heavliy on ttie input from district administrators. At this stage 
evaluators determine what is be evaluated and how the evaluation Is 
to be done. Determining the areas to be evaluated and the types of 
Information that should be collected should be based on the identified 
Information needs of the district and its audiences and the evidence 
available In literature and theory that suggests the inclusion of certain 
areas In the design. From the purposes for which the evaluation Is being 
conducted^ the audiences to whom result^j to be reported, the char- 
acteristics of the area chosen, types of information selected for avalua- 
tion, and the resources available, the specific methodologies used in 
the evaluation are selected* Criteria for judging the results are 
established. Audiences and their information needs for reporting pur- 
poses are specified. And finally p an operational plan Including staff 
requirements and responsibil itles^ resource allocations and a schidule 
of evaluation activities is developed. 

Selecting or Construct in g Instrum ents 

Most evaiuation efforts require the collection of similar Information 
across objects of the evaluation^ e,g. students , teachers, schools ^ etc* 
This necessitates the use of some form of instrumentation whether It be 
checklistSi tests, attitude scales, observation scales, questionnaires, 
opinionnalreSi or forms for record keeping* Prior to the implementation 
of evaluation activities, evaluators must determine instrumentation needs, 
determine criteria for the type of instruments to be used, select 
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appropriat& Instruments or, when no instrumentation is available, devfelop 
the necessary instruments. When instrumints must be developed additional 
tasks of pilot testing, determining reliability and validity, and revising 
instruments are required. 

Collecting Data 

The actual work of gathering evaluative information consists of task$ 
such a$; (1) identifying the information sources for conecting the data^ 
(2) specifying the methods to be used In collecting data; (3) specifying 
sampling procedures; (4) training personnel to collect dataj and (5) admin-^ 
Istering evaluation instruments and recording the data. It is In this 
phase of the evaluation process that the most extensive involvement and 
cooperation of the school district staff Is needed. 

PrQcessIng Data 

Processing data consists of preparing results so that they can be 
analyEied. More specifically it consists of providing a format for coding 
data I scoring instruments and providing for data storage, management and 
retrliesvaK For districts having access to data processing equipment such 
as coriputers^ this responsibility may also include using existing or 
writifng new computer programs. 

Analyzing and Interpretin g Inform ation 

After the results are collected and coded they need to be analyzed 
so that the meaning of the results can be determined. This consists of 
selecting an appropriate analytical procedure, designing a means for 
perfomlng the analysi'^ and performing the analysis. The results are then 
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interpreted in terms of pTfeviously stated criteria, the pMfposes of the 
evaluation and the identified district needs. 

Reportin g Information 

This area of responsibility consists of translatirig the results of 
-^nh'^ilon into comriunl cations appropriate for the audiences who are to 
receive the results. It Includes specific tasks such as' (1) specifying 
means for providing Inforfnation to the audiences; (2) spMlfying the format 
for evaluation reports; {3) developing the communicatlipiris and (4) assisting 
administrators in obtaining feedback. This area of rtspanslbllity must 
necessarily be closely tied to the total district -s wnwuriTcation program. 

The relationship of the primary roles for evaluafcfftir) of administrators 
and evaluators can perhaps be more clearly understood by placing them 
together in an approximate order of their occurrence, 
__ FigufgJ,^ Primary Roles For Evaluation. 

Administrators ^ Eva lu a tors 

1, Establishing a supportive climate 

2, Focusing evaluations on district 
needs 

3, Providing organlxation and support 



10. Cormuni eating reoults to 
audiences 

11 , Utilizing results 

The reader should nott here the use of the word ''pHwry." 
Responsibilities for evalMation, both ideally and practically ^ are not as 
clear cut. Both administrators and evaluators have secQndairy important 
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4, Planinlri.9 evaluations 

5, SfTactlrtg or construct- 
ing tfirtstruments 

6, Collecting data 

7, Prmmtmg data 

8, Ana1y:^1ng and interpret- 
ing dtats^ 

9, Reporting information 
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responsibilities which overlap. Evaluators^ through Inservice programs 
that they conduct for the staff, assist in the establishment of supportive 
climate, communl'Catlng evaluation results m4 providing assistance and 
direction for uttHzIng results. Administrators, through their efforts 
to focus evaluations and the organiiation m4 support they provide ^ assist 
evaluators in planning and Implementing evaluations. 

It has aUo not been the intent of tliis discussion to Imply that 
adrninistrators and evaluators are the onlj^ pmons who have responsi bill tie 
for evaluation in a school district. While it may be granted that admin- 
istrators and ^yaluators have the major responsibilities for install ings 
designing and implementing evaluation activities^ It is most often the 
level of participation by other staff members and constituents that 
determines the quality^ appropriateness aiid^ useful ness of the evaluations 
conducted in a district. Evaluation tliirefore, a team activity 
enhanced by the efforts and ideas brought to Tt by the many persons who 
are knowledgeabTI^ and interested in a school'dlstrict. 
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IMPLICATIONS OF EVALUATION 
FOR SCHOOL DISTRICTS 



Administrators, particularly those who are developing evaluation 

systems in their district for the first times need to be aware of some 

of the implications that evaluation may have for themselves, their staff 

and the programs for which they are responsible. Knowledge of some of 

these possible outcomes may enable them to be better prepared and able 

to deal with some of the change? 'hat are bound to occur. 

A^fnini&trators may fi^id t}iat the utilisation of 
evaluation results vequirQB a ahange in their 
approaah to sohool distriat managemmt. 

The kinds of Information that administrators use as a basis for 

their actions vary considerably. Some are objective and some are not. 

Airaslan (1974, p, 148), for example, describes the following as bases 

for administrative action which are more or less objective: 

Some decisions are made on the basis of authority, 
exemplified by the practice of basing action upon 
the solicited opinions of experts or specialists. 
Other decisions are made on the basis of tradition , 
exemplified by the practice of "letting well enough 
alone" cr adopting the policies of some modern school 
of logical deduction, exemplified by the practice of 
arriving at a course of action by a chain of deductive 
reasoning starting from certain premises or assump- 
tions. Finally^ there are decisions based on empirical 
evidence, exemplified by the practice of arriving 
at solutions on the basis of collected data* 

Equally^ if not more cormion sources for actions are less objective 

and arise out of a political context. The attitude and/or opinions of 

the district's constituency, i.e, local board of education, influential 

or powerful citizens and pressure groups ^ etc, must always be considered 
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by administrators and often becoma the source from which their decisions 
are made. 

Administrators, as a result of their philosophy, training and/or 
experience, tend to choose as a basis for their actions the source or 
sources which have proven to be most effective for them. As a result, 
for some the administration of a school district will be politically 
based. For others, administration will be an active process of utilizing 
the most expedient from a variety of sources. And for a few, adminis- 
tration will primarily be based on some objective inforinatlon or facts. 

Extensive utilization of evaluation results is consistent v/1th an 

administrative phiiosophy which prefers administrative to political 

tasks and views administration as the efficient use of resources (House ^ 

1973, p. 4). Administrators who do not currently have this philosophy 

toward their work and establish an ef^^^ctive evaluation program may 

experience changes in their approach ld the management and day-to-day 

operation of their district. 

Evaluation in sahooZ distriats may lead to an 
inar&aBBd awareness by staff and publios of what 
is happening in Bahoots. 

Parents of school children have a pretty thorough knowledge of 

what IS going on in the classes their children attend. School building 

staffs have a fair picture of what is happening In their school* Too 

few people, educators and publics alike, however ^ have a broader knowledge 

of what is happening in the schools. Evaluations provide an opportunity 

for a review of school district programs and their effects and the 
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dissemination of this information to their constituents. This should 

result 1n a more knowledgeable public and a greater understanding by 

them of the problems faced by administrators. 

The velease of evaluation resuttB may l&ad to 
more overt aonftiats regarding the opi?ration 
of sohool districts t 

The provision of objective^ evaluative information ma: ivide 

fuel for opposing views creating stronger disagreements amony factions 

supporting different directions for school district operations, A 

primary example of this type of situation has occurred with the release 

of information indicating that SAT scores of high school students have 

dropped. Some groups quickly used the Information to enhance their 

argument against forced busing. Others used the information to indicate- 

an abdication of educational responsibH ities by the parents. Even when 

results are not used to support a given position^ they can and often 

are used as a source of criticism of the schools ^ their personnel and 

programs. 

The aon^at of evaluation in sah^ol distriats 
may lead to ahanges i>n ^ ration and programs, 

Frlflrjson (1973s p^ 21) not .nat ''If the evaluation Is to assess 

all the pertinent features of a school prog^^^.m. the admlnistrdtor must 

keep 1n mind that he^ and some of the program's features that h values 

most, may be fair game for the evaluation," Evaluations that identify 

weaknesses in the school program can be expected to create pressures for 

change. Administrators and staff menibers who are not prepared or willing 
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to make some changes as a result of the evaluations they conduct should 
not be Surprised if increaseddlssatisfaction with the school results* 

The colteatio7ij use mid 2^&'Jeaa^ of evaluation 
infommtion may apBate staff unrest. 

Whether it is fear of what evaluation results may reveal or fear of 

inappropriate uses of evaluation for personnel assessment, many staff 

members and particularly teachers are resistant to evaluations of thern= 

selves or their programs. This resistance is still evident when efforts 

are made to assure staff that the results will not be used for personnel 

assessment. Obviously, school districts need fair, objective methods of 

evaluating programs and their staff. But It would be unrealistic to 

assume that the implementation of such a program will not meet with 

soma staff resistance- 

The utiliBation of evaiuation TBSultB may lead to more 
Bffiaient distiHat operation and mofe effeGtive. Bduoa^ 
tional exp&ri&naes far stud&nts. 

Those who advocate the use of evaluation in school districts most 
often base their arguments on the benefits that an administrato)^ and his 
programs can accrue. The types of benefits that they suggest aro certainly 
Impressive and give credence to the suppositions that the risks of con- 
ducting evaluation are worth taking. Stufflebeam (1973, pp. 6=8)5 for 
example^ lists the following as hypotheses about the effects of evaluation 
in a school district. 

A. Improvement of S tudent Learning 

1, improved general educational development of 
students, 

2. better identification and treatment of 
students* special learning needSs 

3, improved student attitudes toward themselves 
and others, and 

4. advanced occupational awareness and maturity. 
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B. Curriculum Development 

1. greater numbers of curriculum changes, 

2. better staff understanding of student 
needs, and 

3. a better match between the curriculu'-n and 
Identified student needs. 

C . Improvement o f Sc h ool Administration 

U clearer more up-to-date^ and more 
defen&iule program priorities, 

2. expenditures that are more in line vnth 
established priorities ^ 

3. better pliLtining, 

4. more efficient operations, and 

5. evidence that special projects are more 
often continued or discontinued based on 
cost/effectiveness data as opposed to the 
availability of external funds. 

D . F inancial Gains 

1. better success in obtaining state and 
federal discretionary funds ^ / 

2. increased money from local operating 
levies and bond issues i and 

3. increased savings through elimination of 
bad programs. 

Improved Comriunication 

1. staff throughout the school system who are 
more conversant with the results of experi- 
mental programs, 

2. community personnel who are better able to 
discuss the quality of school programs based 
on empirical evidence^ 

3. state officials who are more knowledgeable of 
the school system's ncval practices and ivho are 
better able to discuss the quality of the school 
practices based on empirical evidence, 

4. more successful projects being integrated into 
the district's regular program, 

5. increased instances of other school districts 
adopting this school system's ideas and programs^ 
and 

6. Increased instances of this particular district 
adopting succgssful ideas and programs of other 
districts. 
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F. increased Public Suppont for Education 

1. more pubiic expressions of confidence in and 
respect for the school system, 

2, a better percentage of bond Issues and oper- 
ating levies passlngt 

3. a public that 1s more conversant with student 
achievement findingSj 

4, a public that is more conversant with the 
results of project evaluations , and 

6. a public that Increasingly judges the school 
school system to be accountable for .s actions. 

It would be ill -advised and inaccurate for this author to suggest 
that only good things will come from the use of evaluation in a school 
district. There are risks involved as in any new venture. But as 
Morphet, Johns and Reller (1967, p. 531) state, "Every social system must 
have the benefit of appraisals if it is to survive, or grew beyond sur- 
vival , " 

Perhaps the most meaningful way to support the use of evaluation in 
a school district is to solicit the opinion of an administrator who has 
had experience with evaluation In his district* Dr. Jack ^aylor (1975\ 
p, 19)5 Superintendent of Saginaw, Michigan public schools states, "I 
don't really know how many potential problems our evaluative feedback 
has helped detect and how many crises have been avoided as rK^u^t^ but 
I do know I wouldn't want to go back to the old ways and find nut..'' 
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